Mo GPU? Mo problem, as long as you have a YARN cluster and good sense of what the entire thing is going on.
spark-submit –master yarn –properties-file bigdl/conf/spark-bigdl.conf –py-files bigdl-0.4.0-SNAPSHOT-python-api.zip,textclassifier.py –archives BigDL_venv.zip –driver-cores 2 –driver-memory 4g –executor-cores 1 –num-executors 128 –total-executor-cores 128 –executor-memory 4g textclassifier.py
Reference(s)