• Non ci sono risultati.

Spark Streaming

N/A
N/A
Info
Scarica
Protected

Academic year: 2021

Condividi "Spark Streaming"

Copied!
5
0
0

Caricamento.... (Visualizza il testo completo ora)

Scarica adesso ( 5 pagine )

Testo completo

(1)

Spark Streaming

(2)

Predict the variety of iris plants in real-time

Inputs:

Training data

▪ Static input file: training.csv

New data

▪ A stream of records about new iris plants

Output

Predicted class label/variety for each new iris plant using only column “sepallength” and

“sepalwidth”

(3)

Training data schema

sepallength: double

sepalwidth: double

petallength: double – not considered

petalwidth: double – not considered

variety: integer

▪ 1 -> Setosa category

▪ 2 -> Versicolor category

▪ 3 -> Virginica category

(4)

New streaming data schema

sepallength: double

sepalwidth: double

petallength: double

petalwidth: double

variety: it is always null for the new data

(5)

Example training data

sepallength,sepalwidth,petallength,petalwidth,variety 5.1,3.5,1.4,0.2,1

Example new streaming data

5.2,3.2,1.4,0.2,

Riferimenti

Scarica adesso ( PDF - 5 pagine - 166.31 KB )

Documenti correlati

L’evoluzione e il ruolo delle professioni economico-contabili nel periodo 1906-1934

Contestualmente la rivoluzione zappiana produsse forti echi oltre che sugli studi aziendali anche sulla prassi professionale (Canziani, 1987, pp. Non vi è dubbio che l’evoluzione

A regola d'arte. Letteratura e dissimulazione religiosa dell'Accademia ferrarese degli Elevati (1540-41)

bivalenza, il dialogo lasciava però trapelare sia il filoerasmismo sia la posizione ferocemente anticlericale dell’autore, sostanziata anche dagli ideali etici e civili laici,

Spark programs are executed (submitted) by using the spark-submit command

spark-submit --class it.polito.spark.DriverMyApplication -- deploy-mode client --master local

Spark SQL is the Spark component for structured data processing

 The DataFrameReader class (the same we used for reading a json file and store it in a DataFrame) provides other methods to read many standard (textual) formats and read data

Spark Streaming is a framework for large scale stream processing

 Split the input stream in batches of 10 seconds each and print on the standard output, for each batch, the occurrences of each word appearing in the batch. ▪ i.e., execute the

Spark Streaming

 If a stock satisfies the conditions multiple times in the same batch, return the stockId only one time for each

Spark SQL is the Spark component for structured data processing

 The DataFrameReader class (the same we used for reading a json file and store it in a DataFrame) provides other methods to read many standard (textual) formats and read data

Spark Streaming

 For each unlabeled sentence the predicted class label value by using a logistic regression algorithm.  You must train the model by using as input two