Deploying hadoop/spark cluster fails following tutorial #112
Description
Hi,
Trying to follow the instructions to deploy a hadoop cluster and I run into this error:
pip install -e .
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop support for Python 2.7 in January 2021. More details about Python 2 support in pip can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support pip 21.0 will remove support for this functionality.
Obtaining file:///home/dan/elasticluster
ERROR: Command errored out with exit status 1:
command: /home/dan/venv/bin/python2 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/dan/elasticluster/setup.py'"'"'; __file__='"'"'/home/dan/elasticluster/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-HxI4r4
cwd: /home/dan/elasticluster/
Complete output (7 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/dan/elasticluster/setup.py", line 39, in <module>
from ez_setup import use_setuptools
File "ez_setup.py", line 30, in <module>
from urllib.request import urlopen
ImportError: No module named request
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
This seems to be an issue with using python2 to install a python3 script, so I recreated the virtual env with python3.
But after installing in a python3 env (successfully), launching the hadoop fails with the following error:
elasticluster start hadoop -n cluster
Traceback (most recent call last):
File "/home/dan/venv/bin/elasticluster", line 33, in <module>
sys.exit(load_entry_point('elasticluster', 'console_scripts', 'elasticluster')())
File "/home/dan/venv/bin/elasticluster", line 25, in importlib_load_entry_point
return next(matches).load()
File "/home/dan/venv/lib/python3.9/importlib/metadata.py", line 77, in load
module = import_module(match.group('module'))
File "/home/dan/venv/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 855, in exec_module
File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
File "/home/dan/elasticluster/elasticluster/__main__.py", line 35, in <module>
import cli.app
File "/home/dan/venv/lib/python3.9/site-packages/cli/app.py", line 242
except Exception, e:
^
SyntaxError: invalid syntax
Which seems like another compatibility error, this time python3 interpreter reading python2 code.