Repository navigation
python-memcached should fail gracefully when the server does not exist. #92
Description
I recently ran into a situation where our build server did not have a running instance of memcached. I would have expected things to fail gracefully. (Clearly, there wouldn't be any caching happening. Each request would simply have to re-fetch any cached data.) But instead of failing gracefully, the caching code failed after the connect attempt to the memcached server failed.
It looks like once the server is marked dead there are some additional calls to cmemcache_hash that get made with strings as keys. To reproduce this, find a server without a memcached instance and try this:
python
Python 3.5.1 (default, Apr 6 2016, 16:48:42)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import memcache
>>> cache = memcache.Client(['127.0.0.1:11211'], debug=1)
>>> cache.set("working?",1,1)
MemCached: MemCache: inet:127.0.0.1:11211: connect: [Errno 111] Connection refused. Marking dead.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 700, in set
return self._set("set", key, val, time, min_compress_len, noreply)
File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 983, in _set
server, key = self._get_server(key)
File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 413, in _get_server
serverhash = serverHashFunction(str(serverhash) + str(i))
File "/<snipped>/lib/python3.5/site-packages/memcache.py", line 65, in cmemcache_hash
(((binascii.crc32(key) & 0xffffffff)
**TypeError: a bytes-like object is required, not 'str'**
So, I temporarily "fixed" the issue with something like this:
--- /<snipped>/lib/python3.5/site-packages/memcache.py 2016-04-07 17:23:41.428028602 -0500
+++ /<snipped>/lib/python3.5/site-packages/memcache.py_fixed 2016-04-07 17:23:13.606003863 -0500
@@ -61,6 +61,9 @@
def cmemcache_hash(key):
+ if isinstance(key, six.text_type):
+ key = key.encode('utf-8')
+
return (
(((binascii.crc32(key) & 0xffffffff)
>> 16) & 0x7fff) or 1)
I'm not sure if this is the right approach to correct the issue or not. Hopefully we can get back to a point where python-memcached can fail gracefully when the underlying service is absent.
Thanks,
Matthew Lee
Activity