You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm highly interested in this patch. Please let me know if there's anything I can do to help. For now, I'll just fork this, apply #78 and regenerate the bindings with a nightly from Thrift.
Ugh, I didn't actually want to link these two issues, but I just wanted to indicate where I moved my comments...
Anyway, so I used Thrift 1.0.0-dev to gain Python 3 support and one of the tests fails:
======================================================================
ERROR: tests.test_api.test_atomic_counters
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/home/jvlek/dev/lib/happybase/tests/test_api.py", line 172, in test_atomic_counters
table.counter_set(row, column, 0)
File "/home/jvlek/dev/lib/happybase/happybase/table.py", line 540, in counter_set
self.put(row, {column: pack_i64(value)})
File "/home/jvlek/dev/lib/happybase/happybase/table.py", line 436, in put
batch.put(row, data)
File "/home/jvlek/dev/lib/happybase/happybase/batch.py", line 132, in __exit__
self.send()
File "/home/jvlek/dev/lib/happybase/happybase/batch.py", line 55, in send
self._table.connection.client.mutateRows(self._table.name, bms, {})
File "/home/jvlek/dev/lib/happybase/happybase/hbase/Hbase.py", line 1598, in mutateRows
self.send_mutateRows(tableName, rowBatches, attributes)
File "/home/jvlek/dev/lib/happybase/happybase/hbase/Hbase.py", line 1607, in send_mutateRows
args.write(self._oprot)
File "/home/jvlek/dev/lib/happybase/happybase/hbase/Hbase.py", line 7470, in write
iter351.write(oprot)
File "/home/jvlek/dev/lib/happybase/happybase/hbase/ttypes.py", line 589, in write
iter6.write(oprot)
File "/home/jvlek/dev/lib/happybase/happybase/hbase/ttypes.py", line 495, in write
oprot.writeString(self.value)
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/thrift/protocol/TBinaryProtocol.py", line 122, in writeString
encoded = bytearray(str, 'utf-8')
nose.proxy.TypeError: encoding or errors without a string argument
Which simplifies to:
>>> from struct import Struct
>>> bytearray(Struct('>q').pack(0), 'utf-8')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: encoding or errors without a string argument
This is due to a new line in TBinaryProtocol that wasn't there before:
It seems that the 'utf-8' encoding parameter in TBinaryProtocol causes the problems, which is new in 1.0.0-dev:
The integer probably should be send as a TType.I64. Does that mean a new message type, because it can't be send as part of a mutateRows_args anymore? Or is there a way to enforce a different datatype with mutations? edit: No, in ttypes.Mutation.thrift_spec a value is defined as TType.STRING
Furthermore, the tearDown fails:
======================================================================
ERROR: test suite for <module 'tests.test_api' from '/home/jvlek/dev/lib/happybase/tests/test_api.py'>
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/suite.py", line 229, in run
self.tearDown()
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/suite.py", line 352, in tearDown
self.teardownContext(ancestor)
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/suite.py", line 368, in teardownContext
try_run(context, names)
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/util.py", line 471, in try_run
return func()
File "/home/jvlek/dev/lib/happybase/tests/test_api.py", line 76, in teardown_module
connection.delete_table(TEST_TABLE_NAME, disable=True)
File "/home/jvlek/dev/lib/happybase/happybase/connection.py", line 320, in delete_table
if disable and self.is_table_enabled(name):
File "/home/jvlek/dev/lib/happybase/happybase/connection.py", line 351, in is_table_enabled
return self.client.isTableEnabled(name)
File "/home/jvlek/dev/lib/happybase/happybase/hbase/Hbase.py", line 727, in isTableEnabled
return self.recv_isTableEnabled()
File "/home/jvlek/dev/lib/happybase/happybase/hbase/Hbase.py", line 744, in recv_isTableEnabled
raise x
thrift.Thrift.TApplicationException: Negative length: -2147418111
The tear down failure also caused an NPE at the server's side:
2015-04-21 17:10:21,018 ERROR [thrift-worker-0] thrift.ProcessFunction: Internal error processing isTableEnabled
java.lang.NullPointerException
at org.apache.hadoop.hbase.util.Bytes.getBytes(Bytes.java:968)
at org.apache.hadoop.hbase.thrift.ThriftServerRunner$HBaseHandler.isTableEnabled(ThriftServerRunner.java:675)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.hbase.thrift.HbaseHandlerMetricsProxy.invoke(HbaseHandlerMetricsProxy.java:67)
at com.sun.proxy.$Proxy6.isTableEnabled(Unknown Source)
at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$isTableEnabled.getResult(Hbase.java:3697)
at org.apache.hadoop.hbase.thrift.generated.Hbase$Processor$isTableEnabled.getResult(Hbase.java:3681)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hadoop.hbase.thrift.TBoundedThreadPoolServer$ClientConnnection.run(TBoundedThreadPoolServer.java:289)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
This code will handle unicode text only, but I think writeString() is supposed to write byte strings. Additionally, this code (ab)uses a bytearray to encode a unicode string into bytes, while a simple text.encode('utf-8') would have sufficed.
Anyway, the problem is the buggy Thrift code: it needs to make up its mind about unicode text versus bytes. It should handle byte strings.
Traceback (most recent call last):
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/home/jvlek/dev/lib/happybase/tests/test_api.py", line 360, in test_scan
list(table.scan(batch_size=None))
File "/home/jvlek/dev/lib/happybase/happybase/table.py", line 294, in scan
if batch_size < 1:
TypeError: unorderable types: NoneType() < int()
In Python 3, comparison with None raises a TypeError. Which in the context of table.scan() makes sense, because batch_size should be an integer >= 1. The TypeError conveys more precise information, without having to change any code. So I suggest updating test_scan() as follows:
Traceback (most recent call last):
File "/home/jvlek/.virtualenvs/happybase/lib/python3.4/site-packages/nose/case.py", line 198, in runTest
self.test(*self.arg)
File "/home/jvlek/dev/lib/happybase/tests/test_util.py", line 34, in check
s = s_hex.decode('hex')
AttributeError: 'str' object has no attribute 'decode'
This one seems like an architectural decision. Changing the decode() call for codec.decode(s_hex, 'hex'), led me to a "TypeError: ord() expected string of length 1, but int found".
util.str_increment() says its intent is to increment and truncate a byte string, so it would be enough to remove the call to ord(), since s[i] would already result in an integer. However, then other tests start to fail, because, for example, the row_prefix for table.scan is not a byte string.
Now, I guess you don't want to convert all strings to byte strings, yet the str_increment is written with byte strings in mind, and assumes a hexadecimal space. However, HBase's row keys are UTF-8 encoded, so I reckon there's no need to wrap around after '\xFF'.
Please find my proposed fix below. All the tests pass, but I don't know if there are any side-effects I'm overlooking, or whether you have a completely different direction in mind.
## happybase/util.py ##MAX_UNICODE=chr(1114111)
defstr_increment(s):
"""Increment and truncate a string (for sorting purposes) This functions returns the shortest string that sorts after the given string when compared using regular string comparison semantics. This function increments the last character that is smaller than MAX_UNICODE, and drops everything after it. If the string only contains MAX_UNICODE characters, `None` is returned. """foriinrange(len(s) -1, -1, -1):
ifs[i] !=MAX_UNICODE:
returns[:i] +chr(ord(s[i]) +1)
returnNone## tests/test_util.py ##deftest_str_increment():
defcheck(s, expected):
v=util.str_increment(s)
assert_equal(expected, v)
assert_less(s, v)
test_values= [
('00', '01'),
('01', '02'),
('fe', 'ff'),
('1234', '1235'),
('12fe', '12ff'),
('12ff', '12fg'),
('424242'+util.MAX_UNICODE, '424243'),
('4242'+2*util.MAX_UNICODE, '4243'),
]
assertutil.str_increment(util.MAX_UNICODE*4) isNonefors, expectedintest_values:
yieldcheck, s, expected
so it would be enough to remove the call to ord(), since s[i] would already result in an integer
That will break on Python 2, since b'foo'[0] will be b'f' on Python 2, and 102 on Python 3. Some compat code is needed here I think. In my Plyvel project I have had the same issue for the same routine.
Although I guess for generating end keys, it still suffices, albeit potentially slightly less efficient (i.e. sending longer end keys than necessary to the server).
That will break on Python 2
Another reason to leave it in (I left the call to ord() in the code).
However, the call to chr() is not backwards compatible, since the equivalent under Python 2 would be unichr(). How did you solve it in your Plyvel project? A quick search on "str_increment" didn't yield anything.
No, it must do the right thing on the byte level, not on the UTF-8 encoding level. This code is used to determine the stop key when using a row prefix, so it must be on the byte level.
unichr() should be avoided; it operates on unicode strings. we're talking bytes here.
In Plyvel the routine is called bytes_increment(), it uses a bytearray under the hood. (That one has PyPy compat issues after compiling it to C code, but that doesn't apply here.)
This encoding compatibility stuff is a tricky business, and I still feel that it involves some architectural decisions, but I'm effectively only using happybase since this week, so I'm either wrong about it involving architectural decisions, or I'm too inexperienced with the library to make them ;)
Below my solution, which suits my needs for now, and which I'll happily replace with happybase > 0.9.
## happybase/util.py ##defstr_increment(s):
"""Increment and truncate a byte string (for sorting purposes) This functions returns the shortest string that sorts after the given string when compared using regular string comparison semantics. This function increments the last byte that is smaller than ``0xFF``, and drops everything after it. If the string only contains ``0xFF`` bytes, `None` is returned. """try:
ba=bytearray(s, "utf-8")
exceptTypeError:
ba=bytearray(s)
foriinrange(len(ba) -1, -1, -1):
ifba[i] !=255:
ba[i] +=1returnba[:(i+1)]
returnNone## tests/test_util.py ##deftest_str_increment():
defcheck(s_hex, expected):
s=codecs.decode(s_hex, 'hex')
v=util.str_increment(s)
v_hex=codecs.encode(v, 'hex')
assert_equal(expected, v_hex)
assert_less(s, v)
test_values= [
(b'00', b'01'),
(b'01', b'02'),
(b'fe', b'ff'),
(b'1234', b'1235'),
(b'12fe', b'12ff'),
(b'12ff', b'13'),
(b'424242ff', b'424243'),
(b'4242ffff', b'4243'),
]
assertutil.str_increment(b'\xff\xff\xff') isNonefors, expectedintest_values:
yieldcheck, s, expected
(test33)[deploy@sfleu1webap11 test33]$ pip install happybase
Downloading/unpacking happybase
Downloading happybase-0.9.tar.gz (62kB): 62kB downloaded
Running setup.py egg_info for package happybase
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/home/deploy/test33/build/happybase/setup.py", line 5, in <module>
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/home/deploy/test33/build/happybase/setup.py", line 5, in <module>
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /home/deploy/test33/build/happybase
Storing complete log in /home/deploy/.pip/pip.log
pip.log:
(test33)[deploy@sfleu1webap11 test33]$ cat /home/deploy/.pip/pip.log
------------------------------------------------------------
/home/deploy/test33/bin/pip run on Thu Feb 4 12:10:22 2016
Downloading/unpacking happybase
Getting page https://pypi.python.org/simple/happybase/
URLs to search for versions for happybase:
* https://pypi.python.org/simple/happybase/
Analyzing links from page https://pypi.python.org/simple/happybase/
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.1.tar.gz#md5=376d9f4d22a1d0d475064e3289de6a92 (from https://pypi.python.org/simple/happybase/), version: 0.1
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.2.tar.gz#md5=b1204c9713612028846629a47d876116 (from https://pypi.python.org/simple/happybase/), version: 0.2
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.3.tar.gz#md5=3b6ea750d8fb83d63fb4f128d11f3e4f (from https://pypi.python.org/simple/happybase/), version: 0.3
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.4.tar.gz#md5=ea87560a8d3946a3fcde6beb3400e395 (from https://pypi.python.org/simple/happybase/), version: 0.4
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.5.tar.gz#md5=ae7de65b85270d54bb05d566ff3b0496 (from https://pypi.python.org/simple/happybase/), version: 0.5
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.6.tar.gz#md5=28905e2a334d6d7b42495f2c5ca46add (from https://pypi.python.org/simple/happybase/), version: 0.6
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.7.tar.gz#md5=655fd08ff83a7132e607176bcee3d0b0 (from https://pypi.python.org/simple/happybase/), version: 0.7
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.8.tar.gz#md5=dc322b9ea8a31121e75ab1d1cd50538c (from https://pypi.python.org/simple/happybase/), version: 0.8
Found link https://pypi.python.org/packages/source/h/happybase/happybase-0.9.tar.gz#md5=0d9ca6e22a7edb03ccd75e99d0acf1ea (from https://pypi.python.org/simple/happybase/), version: 0.9
Using version 0.9 (newest of versions: 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1)
Downloading from URL https://pypi.python.org/packages/source/h/happybase/happybase-0.9.tar.gz#md5=0d9ca6e22a7edb03ccd75e99d0acf1ea (from https://pypi.python.org/simple/happybase/)
Running setup.py egg_info for package happybase
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/home/deploy/test33/build/happybase/setup.py", line 5, in <module>
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/home/deploy/test33/build/happybase/setup.py", line 5, in <module>
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
----------------------------------------
Cleaning up...
Removing temporary dir /home/deploy/test33/build...
Command python setup.py egg_info failed with error code 1 in /home/deploy/test33/build/happybase
Exception information:
Traceback (most recent call last):
File "/home/deploy/test33/lib/python3.3/site-packages/pip/basecommand.py", line 134, in main
status = self.run(options, args)
File "/home/deploy/test33/lib/python3.3/site-packages/pip/commands/install.py", line 236, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/home/deploy/test33/lib/python3.3/site-packages/pip/req.py", line 1134, in prepare_files
req_to_install.run_egg_info()
File "/home/deploy/test33/lib/python3.3/site-packages/pip/req.py", line 259, in run_egg_info
command_desc='python setup.py egg_info')
File "/home/deploy/test33/lib/python3.3/site-packages/pip/util.py", line 670, in call_subprocess
% (command_desc, proc.returncode, cwd))
pip.exceptions.InstallationError: Command python setup.py egg_info failed with error code 1 in /home/deploy/test33/build/happybase
My environment:
(test33)[deploy@sfleu1webap11 test33]$ uname -a && cat /etc/issue && which python && python --version
Linux sfleu1webap11.eu1.ecsna.com 2.6.32-573.12.1.el6.x86_64 #1 SMP Tue Dec 15 21:19:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
CentOS release 6.7 (Final)
Kernel \r on an \m
~/test33/bin/python
Python 3.3.2`
Installed /home/deploy/test33/build/python-jenkins/pbr-1.8.1-py3.3.egg
[pbr] Processing SOURCES.txt
warning: LocalManifestMaker: standard file '-c' not found
warning: no previously-included files found matching '.gitignore'
warning: no previously-included files found matching '.gitreview'
warning: no previously-included files matching '.pyc' found anywhere
in distribution
warning: no previously-included files found matching '.gitignore'
warning: no previously-included files found matching '.gitreview'
warning: no previously-included files matching '.pyc' found anywhere
in distribution
Downloading/unpacking happybase
Downloading happybase-0.9.tar.gz (62kB): 62kB downloaded
Running setup.py egg_info for package happybase
Traceback (most recent call last):
File "", line 16, in
File "/home/deploy/test33/build/happybase/setup.py", line 5, in
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "", line 16, in
File "/home/deploy/test33/build/happybase/setup.py", line 5, in
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
Cleaning up...
Command python setup.py egg_info failed with error code 1 in
/home/deploy/test33/build/happybase
Storing complete log in /home/deploy/.pip/pip.log`
My environment:
`(test33)[deploy@sfleu1webap11 test33]$ uname -a && cat /etc/issue &&
which python && python --version
Linux sfleu1webap11.eu1.ecsna.com 2.6.32-573.12.1.el6.x86_64 #1 SMP
Tue Dec 15 21:19:08 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
CentOS release 6.7 (Final)
Kernel \r on an \m
~/test33/bin/python
Python 3.3.2`
Am I doing something wrong or is this a bug?
Reply to this email directly or view it on GitHub: #40 (comment)
I'm a little confused as to the status of related tickets and known issues. Can you summarize what needs to be done in order to make happybase work with Python 3? If the amount of work is reasonable and I can get permission from my employer, I'm likely willing to do what needs to be done.
I created a pull request for Python 3 support. Note that while I don't think accepting the pull request is a good idea yet (see details in the pull request), it might still be useful for other developers.
I closed the pull request, as most of the Hbase documentation conflicts with the comment in the Hbase.thrift file that suggests that keys, values, etc. should be UTF-8. I'll make another pull request which accepts bytes() instances in Python 3.
For Python3,There is still an error. I used pip to install the happybase.
➜ ~ pip3 install happybase
Collecting happybase
Using cached happybase-0.9.tar.gz
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/tz/dhnmphd97tl_90vhb80c3bgw0000gn/T/pip-build-2ekx3nnz/happybase/setup.py", line 5, in <module>
execfile('happybase/_version.py')
NameError: name 'execfile' is not defined
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/tz/dhnmphd97tl_90vhb80c3bgw0000gn/T/pip-build-2ekx3nnz/happybase/
there's a pr that needs some more work. you could have read my comments over there.
i don't give an eta since i maintain this project in my SPARE time even though i don't use it myself anymore, while pretty much ALL users are corporate users, and only very few of them have ever helped out in any significant way, e.g. by contributing code or providing financial support.
even though i keep my project alive since i care about it, i don't work for free on demand by others, so i only spend time on it when i feel like it.
i also refuse to feel guilty about this. this is a cultural problem with open source projects used in corporate environments. only taking while never giving back is not sustainable.
Activity
wbolster commentedon Jan 13, 2015
Thift seems to be Python 3 compatible nowadyas. See also #78.
wbolster commentedon Jan 13, 2015
(removed comment, now part of issue description)
jeroenvlek commentedon Apr 20, 2015
Thrift is getting there: https://issues.apache.org/jira/browse/THRIFT-1857.
I'm highly interested in this patch. Please let me know if there's anything I can do to help. For now, I'll just fork this, apply #78 and regenerate the bindings with a nightly from Thrift.
jeroenvlek commentedon Apr 22, 2015
Ugh, I didn't actually want to link these two issues, but I just wanted to indicate where I moved my comments...
Anyway, so I used Thrift 1.0.0-dev to gain Python 3 support and one of the tests fails:
Which simplifies to:
This is due to a new line in TBinaryProtocol that wasn't there before:
It seems that the 'utf-8' encoding parameter in TBinaryProtocol causes the problems, which is new in 1.0.0-dev:
The integer probably should be send as a TType.I64. Does that mean a new message type, because it can't be send as part of a mutateRows_args anymore? Or is there a way to enforce a different datatype with mutations? edit: No, in ttypes.Mutation.thrift_spec a value is defined as TType.STRING
Furthermore, the tearDown fails:
The tear down failure also caused an NPE at the server's side:
I will look into the tearDown() exception now.
wbolster commentedon Apr 22, 2015
This code you mentioned handles byte strings on Python 2:
The updated code you mentioned is plain wrong if it is still intended to handle byte strings:
This code will handle unicode text only, but I think
writeString()is supposed to write byte strings. Additionally, this code (ab)uses abytearrayto encode a unicode string into bytes, while a simpletext.encode('utf-8')would have sufficed.Anyway, the problem is the buggy Thrift code: it needs to make up its mind about unicode text versus bytes. It should handle byte strings.
jeroenvlek commentedon Apr 22, 2015
I see now that it is not in Thrift's master, but it was introduced by one of the patches for Python 3 support. I will notify that thread.
jeroenvlek commentedon Apr 22, 2015
In Python 3, comparison with None raises a TypeError. Which in the context of table.scan() makes sense, because batch_size should be an integer >= 1. The TypeError conveys more precise information, without having to change any code. So I suggest updating test_scan() as follows:
jeroenvlek commentedon Apr 22, 2015
This one seems like an architectural decision. Changing the decode() call for codec.decode(s_hex, 'hex'), led me to a "TypeError: ord() expected string of length 1, but int found".
util.str_increment() says its intent is to increment and truncate a byte string, so it would be enough to remove the call to ord(), since s[i] would already result in an integer. However, then other tests start to fail, because, for example, the row_prefix for table.scan is not a byte string.
Now, I guess you don't want to convert all strings to byte strings, yet the str_increment is written with byte strings in mind, and assumes a hexadecimal space. However, HBase's row keys are UTF-8 encoded, so I reckon there's no need to wrap around after '\xFF'.
Please find my proposed fix below. All the tests pass, but I don't know if there are any side-effects I'm overlooking, or whether you have a completely different direction in mind.
wbolster commentedon Apr 22, 2015
That's not true. Row keys are byte strings, and a lot of data models depend on that.
wbolster commentedon Apr 22, 2015
That will break on Python 2, since
b'foo'[0]will beb'f'on Python 2, and102on Python 3. Some compat code is needed here I think. In my Plyvel project I have had the same issue for the same routine.jeroenvlek commentedon Apr 23, 2015
Ah yes, I should have read the follow up comment on this one: http://permalink.gmane.org/gmane.comp.java.hadoop.hbase.user/18006
Although I guess for generating end keys, it still suffices, albeit potentially slightly less efficient (i.e. sending longer end keys than necessary to the server).
Another reason to leave it in (I left the call to ord() in the code).
However, the call to chr() is not backwards compatible, since the equivalent under Python 2 would be unichr(). How did you solve it in your Plyvel project? A quick search on "str_increment" didn't yield anything.
wbolster commentedon Apr 23, 2015
Few quick follow-up notes:
unichr()should be avoided; it operates on unicode strings. we're talking bytes here.bytes_increment(), it uses abytearrayunder the hood. (That one has PyPy compat issues after compiling it to C code, but that doesn't apply here.)jeroenvlek commentedon Apr 23, 2015
This encoding compatibility stuff is a tricky business, and I still feel that it involves some architectural decisions, but I'm effectively only using happybase since this week, so I'm either wrong about it involving architectural decisions, or I'm too inexperienced with the library to make them ;)
Below my solution, which suits my needs for now, and which I'll happily replace with happybase > 0.9.
csabaszilveszter commentedon Feb 4, 2016
Hi,
...pip install happybase doesn't seem to work...
pip.log:
My environment:
Am I doing something wrong or is this a bug?
wbolster commentedon Feb 18, 2016
python 3 is unfortunately not yet supported, see related tickets and known issue list for details
sent from my phone. please forgive my tpyos.
On February 4, 2016 7:06:52 PM GMT+01:00, Csaba Szilveszter notifications@github.com wrote:
surfacepatterns commentedon Feb 27, 2016
I'm a little confused as to the status of related tickets and known issues. Can you summarize what needs to be done in order to make happybase work with Python 3? If the amount of work is reasonable and I can get permission from my employer, I'm likely willing to do what needs to be done.
surfacepatterns commentedon Mar 9, 2016
I created a pull request for Python 3 support. Note that while I don't think accepting the pull request is a good idea yet (see details in the pull request), it might still be useful for other developers.
surfacepatterns commentedon Mar 10, 2016
I closed the pull request, as most of the Hbase documentation conflicts with the comment in the Hbase.thrift file that suggests that keys, values, etc. should be UTF-8. I'll make another pull request which accepts bytes() instances in Python 3.
surfacepatterns commentedon Mar 11, 2016
I've filed a new pull request with code that does the right thing (mainly, expecting bytes() instances for table names, columns, cell values, etc.).
kayzhou commentedon Mar 18, 2016
For Python3,There is still an error. I used pip to install the happybase.
surfacepatterns commentedon Mar 20, 2016
@kayzhou: Python 3 compatibility has not yet made its way into the master branch.
ssaamm commentedon Apr 22, 2016
Is there a recommendation for users who want to use happybase in Python 3?
scttcper commentedon Apr 22, 2016
@ssaamm
use the python3 branch.
requirements.txt
git+https://github.com/wbolster/happybase.git@python3csabaszilveszter commentedon Apr 23, 2016
Niiiiiiice.... thx
On Fri, Apr 22, 2016 at 8:46 PM, Scott Cooper notifications@github.com
wrote:
ManikandanUV commentedon May 18, 2016
Is there an ETA on when python 3 support will hit the master?
wbolster commentedon May 20, 2016
warning. rant ahead. this is not personal.
there's a pr that needs some more work. you could have read my comments over there.
i don't give an eta since i maintain this project in my SPARE time even though i don't use it myself anymore, while pretty much ALL users are corporate users, and only very few of them have ever helped out in any significant way, e.g. by contributing code or providing financial support.
even though i keep my project alive since i care about it, i don't work for free on demand by others, so i only spend time on it when i feel like it.
i also refuse to feel guilty about this. this is a cultural problem with open source projects used in corporate environments. only taking while never giving back is not sustainable.
wbolster commentedon Aug 1, 2016
i merged #116 into master