• Non ci sono risultati.

Cluster BigData@Polito cluster

N/A
N/A
Info
Scarica
Protected

Academic year: 2021

Condividi "Cluster BigData@Polito cluster"

Copied!
5
0
0

Caricamento.... (Visualizza il testo completo ora)

Scarica adesso ( 5 pagine )

Testo completo

(1)
(2)

BigData@Polito cluster

Cluster node 1

Cluster

Cluster node N Cluster

node 2

bigdatalab.polito.it server

(gateway)

User with personal computer

ssh

HDFS Your “local” home is

on a shared area Web

browser

jupyter.polito.it server (gateway)

User with personal computer

(3)

Jupyter notebook

Browser-based interactive IDE

Specific “notebooks” can be used to run Spark applications on the Spark cluster

PySpark (Local)

▪ Run the application in a container (in a local instance of Spark)

PySpark (Yarn)

▪ Run the application on the BigData@Polito cluster

Both notebooks read/write data from/in HDFS

(4)

Copy the input data of your application from the local drive of your personal workstation on the HDFS file system of the cluster

Open an interactive PySpark shell by using a Jupyter notebook

Write the python/spark code you want to

execute and execute it step-by-step by using the PySpark notebook

The result is stored in the output HDFS folder specified in your application

(5)

Create a PySpark Jupyter notebook in the local file system of the machine hosting Jupyter (jupyter.polito.it)

Riferimenti

Scarica adesso ( PDF - 5 pagine - 424.61 KB )

Documenti correlati

La transizione tra interno ed esterno nell'architettura contemporanea

Contesto e spazio architettonico è un libro di Giovanna Grella pubblicato da Gangemi nella collana Arti visive, architettura e urbanistica: acquista su IBS a 20.40€.. Gli

Photometric determination of the age and distance of the open cluster NGC 2420

By comparing our data with isochrones, theoretical models describing the position of stars with the same age in CMDs, we were able to determine the age, the color excess due to

Execution of MapReduce applications

 Copy the jar file containing your application from your personal workstation (or the workstation of LABINF) in the “local” file system of the gateway. ▪ Drag and Drop from your

Small Noise Asymptotic Expansion for a Infinite Dimensional Stochastic Reaction-Diffusion Forced Van Der Pol Equation

3 we will state the results which show the validity of the asymptotic expansion in powers of a small parameter ε, together with the existence and uniqueness for the solution of

Chronic bronchitis without airflow obstruction, asthma and rhinitis are differently associated with cardiovascular risk factors and diseases

The associations of respiratory diseases with reported cardiovascular risk factors (lifestyles, hypertension, dyslipidaemia), heart disorders (myocardial infarction,

Comparing data assimilation filters for parameter estimation in a neuron model

In this paper, we apply the EnKF, the BF and the OPT-SIRS to the estimation and prediction of a single computational neuron model with ten parameters and conduct a comparison study

TIVA con Fentanyl o Remifentanil in chirurgia breve

Fentanyl e Remifentanil sono due tra gli oppioidi più utilizzati in ambito anestesiologico ai fini del sollevamento dal dolore, obiettivo che costituisce una priorità

Strain engineering of germanium surface states: an ab-initio study.

Once determined the best working strategy, we have studied in chapter 8 the effects of the strain, both in biaxial or uniaxial form, on the structural and electronic properties of