[metasim]

Aims to integrate Python setuptools build with sbt to create a binary artifact sufficient for pyspark deployment. Removes dependency on sbt-spark-packages, which is languishing.

Misc Resources

• https://setuptools.readthedocs.io/en/latest/setuptools.html#id4
• https://github.com/pypa/sampleproject
• https://docs.python.org/3.7/distutils/setupscript.html
• https://docs.python.org/2/distutils/setupscript.html#distutils-additional-files
• https://www.python.org/dev/peps/pep-0427/
• https://jichu4n.com/posts/how-to-add-custom-build-steps-and-commands-to-setuppy/
• https://github.com/anguenot/pyspark-cassandra
• https://bytes.grubhub.com/managing-dependencies-and-artifacts-in-pyspark-7641aa89ddb7
• https://github.com/apache/spark/blob/master/python/setup.py
• https://github.com/locationtech/geomesa/blob/master/geomesa-spark/geomesa_pyspark/pom.xml

[metasim]

This doesn't yet work. Needs to set up the pyspark jar paths and test directories.

[bguseman]

Without fully grokking all the details, this looks solid. Will a pip install pull in everything that's needed including jvm stuff?

[metasim]

Not sure about pulling in the jvm stuff. I'm wondering if something in the __init__.py might be required.

[metasim]

Replaced by locationtech#125