Skip to content

UnicodeDecodeError in python-mpd2 (on Windows 10) #93

Open
Open
UnicodeDecodeError in python-mpd2 (on Windows 10)#93

Description

@handsomegui
handsomegui
opened on Feb 23, 2018

Hi, i found a strange problem, here below is the info:

OS: Windows 10 Pro
MPD: 0.20.16 https://www.musicpd.org/download/win32/0.20.16/mpd.exe
python: 2.7.14 64bit
python-mpd2: 0.5.5

handler = mpd.MPDClient(use_unicode=True)
handler.timeout = 10
handler.connect("127.0.0.1", 6600)

# ...

artistList = handler.list('Artist')

Traceback (most recent call last):
  File "C:/Users/zhenwei/.code/replayer/replayer.py", line 5707, in initUI
    artistList = handler.list('Artist')
  File "C:\Python27\lib\site-packages\mpd.py", line 629, in decorator
    return wrapper(self, name, args, bound_decorator(self, returnValue))
  File "C:\Python27\lib\site-packages\mpd.py", line 254, in _execute
    return retval()
  File "C:\Python27\lib\site-packages\mpd.py", line 623, in decorator
    return function(self, *args, **kwargs)
  File "C:\Python27\lib\site-packages\mpd.py", line 398, in _fetch_list
    return self._wrap_iterator(self._read_list())
  File "C:\Python27\lib\site-packages\mpd.py", line 374, in _wrap_iterator
    return list(iterator)
  File "C:\Python27\lib\site-packages\mpd.py", line 318, in _read_list
    for key, value in self._read_pairs():
  File "C:\Python27\lib\site-packages\mpd.py", line 314, in _read_pairs
    pair = self._read_pair(separator)
  File "C:\Python27\lib\site-packages\mpd.py", line 302, in _read_pair
    line = self._read_line()
  File "C:\Python27\lib\site-packages\mpd.py", line 284, in _read_line
    line = decode_str(line)
  File "C:\Python27\lib\site-packages\mpd.py", line 33, in decode_str
    return s.decode("utf-8")
  File "C:\Python27\lib\encodings\utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc0 in position 13: invalid start byte

Any help is warmly appreciated.

Activity

  1. rnixx commented on Feb 23, 2018

    @rnixx
    rnixx
    on Feb 23, 2018
    Contributor

    Do you have the same problem when using master instead of 0.5.5?

  2. Mic92 commented on Feb 23, 2018

    @Mic92
    Mic92
    on Feb 23, 2018
    Owner

    You can also test 1.0.0 instead, that was added some days ago.
    If you could dump traffic with wireshark it would be also helpful.

  3. Mic92 commented on Feb 23, 2018

    @Mic92
    Mic92
    on Feb 23, 2018
    Owner

    Is the bug similar to what you have been discovered before for files? #49

  4. handsomegui commented on Feb 23, 2018

    @handsomegui
    handsomegui
    on Feb 23, 2018
    Author

    Do you have the same problem when using master instead of 0.5.5?

    Yes. After i pip install --upgrade python-mpd2==1.0.0

    The problem remain :(

  5. handsomegui commented on Feb 23, 2018

    @handsomegui
    handsomegui
    on Feb 23, 2018
    Author

    You can also test 1.0.0 instead, that was added some days ago.
    If you could dump traffic with wireshark it would be also helpful.

    I will try to learn wireshark tool usage. Thank you.

  6. handsomegui commented on Feb 23, 2018

    @handsomegui
    handsomegui
    on Feb 23, 2018
    Author

    Is the bug similar to what you have been discovered before for files? #49

    I guess it might be the same issue.
    If this is true, then the problem was from the MPD upstream.

  7. handsomegui commented on Feb 24, 2018

    @handsomegui
    handsomegui
    on Feb 24, 2018
    Author

    New found:

    After i move away a folder(which contains 825 files) from music_directory to a place outside, the above problem disappeared.

    So it should be caused by some special characters/encoding in the artist meta info.

  8. handsomegui commented on Feb 24, 2018

    @handsomegui
    handsomegui
    on Feb 24, 2018
    Author

    I also report this issue to MPD repo.

    MusicPlayerDaemon/MPD#231

  9. Mic92 commented on Feb 24, 2018

    @Mic92
    Mic92
    on Feb 24, 2018
    Owner

    @handsomegui you should provide MPD more specific information, what files causes an error.
    Otherwise they will not be able to fix this.

  10. handsomegui commented on Feb 28, 2018

    @handsomegui
    handsomegui
    on Feb 28, 2018
    Author

    @Mic92 Thanks for the hint.

    I have a lot of files under the music_directory.
    I have no clue to find out the file which contained the error encoding artist info.

    I try to print out the unable decoded string python-mpd2 reported by add one print line in the base.py file as below image.

    base

    It shows the error decoded artist info as below:_

    DEBUG INFO: OK

DEBUG INFO: Artist: 

DEBUG INFO: Artist: ????

DEBUG INFO: Artist: ????????????_??????????¼??

DEBUG INFO: Artist: ????????л??˳??

DEBUG INFO: Artist: ??????ȫ???ݳ?

DEBUG INFO: Artist: ?????��????ƽ?

'utf8' codec can't decode byte 0xc0 in position 13: invalid start byte

    Could someone help me to find out the file which caused the error?
    Thanks.

  11. Mic92 commented on Feb 28, 2018

    @Mic92
    Mic92
    on Feb 28, 2018
    Owner

    You can track this down using:

    def decode_str(s):
    try:
        return s.decode("utf-8")
    except Exception as e:
        import pdb; pdb.set_trace()

    then you can print s to see what is going wrong.

  12. handsomegui commented on Feb 28, 2018

    @handsomegui
    handsomegui
    on Feb 28, 2018
    Author

    @Mic92 Thank you very much.

    Here below is what i got:

    --Return--
> c:\python27\lib\site-packages\mpd\base.py(52)decode_str()->None
-> import pdb; pdb.set_trace()
(Pdb) s
> c:\python27\lib\site-packages\mpd\base.py(528)_read_line()
-> if not line.endswith("\n"):
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(528)_read_line()
-> if not line.endswith("\n"):
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(528)_read_line()->None
-> if not line.endswith("\n"):
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(548)_read_lines()
-> line = self._read_line()
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(548)_read_lines()->None
-> line = self._read_line()
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(226)_parse_pairs()
-> for line in lines:
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(226)_parse_pairs()->None
-> for line in lines:
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(286)_parse_list()
-> for key, value in self._parse_pairs(lines):
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(286)_parse_list()->None
-> for key, value in self._parse_pairs(lines):
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(567)_wrap_iterator()
-> return list(iterator)
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(567)_wrap_iterator()->None
-> return list(iterator)
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(378)command_callback()
-> res = self._wrap_iterator(res)
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(378)command_callback()->None
-> res = self._wrap_iterator(res)
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(480)_execute()
-> return retval()
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(480)_execute()->None
-> return retval()
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\python27\lib\site-packages\mpd\base.py(388)mpd_command()
-> return wrapper(self, name, args, callback)
(Pdb) s
--Return--
> c:\python27\lib\site-packages\mpd\base.py(388)mpd_command()->None
-> return wrapper(self, name, args, callback)
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\users\zhenwei\.code\replayer\replayer.py(5744)initUI()
-> artistList = handler.list('Artist')
(Pdb) s
--Return--
> c:\users\zhenwei\.code\replayer\replayer.py(5744)initUI()->None
-> artistList = handler.list('Artist')
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\users\zhenwei\.code\replayer\replayer.py(5727)__init__()
-> self.initUI(self.font)
(Pdb) s
--Return--
> c:\users\zhenwei\.code\replayer\replayer.py(5727)__init__()->None
-> self.initUI(self.font)
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\users\zhenwei\.code\replayer\replayer.py(1376)init_ui()
-> self.libraryView = Library(self.fontValue)
(Pdb) s
--Return--
> c:\users\zhenwei\.code\replayer\replayer.py(1376)init_ui()->None
-> self.libraryView = Library(self.fontValue)
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\users\zhenwei\.code\replayer\replayer.py(548)__init__()
-> self.init_ui()
(Pdb) s
--Return--
> c:\users\zhenwei\.code\replayer\replayer.py(548)__init__()->None
-> self.init_ui()
(Pdb) s
AttributeError: "'NoneType' object has no attribute 'endswith'"
> c:\users\zhenwei\.code\replayer\replayer.py(6749)<module>()
-> mw = MW()
(Pdb) s
--Return--
> c:\users\zhenwei\.code\replayer\replayer.py(6749)<module>()->None
-> mw = MW()
(Pdb) s
Traceback (most recent call last):
  File "C:/Users/zhenwei/.code/replayer/replayer.py", line 6749, in <module>
    mw = MW()
  File "C:/Users/zhenwei/.code/replayer/replayer.py", line 548, in __init__
    self.init_ui()
  File "C:/Users/zhenwei/.code/replayer/replayer.py", line 1376, in init_ui
    self.libraryView = Library(self.fontValue)
  File "C:/Users/zhenwei/.code/replayer/replayer.py", line 5727, in __init__
    self.initUI(self.font)
  File "C:/Users/zhenwei/.code/replayer/replayer.py", line 5744, in initUI
    artistList = handler.list('Artist')
  File "C:\Python27\lib\site-packages\mpd\base.py", line 388, in mpd_command
    return wrapper(self, name, args, callback)
  File "C:\Python27\lib\site-packages\mpd\base.py", line 480, in _execute
    return retval()
  File "C:\Python27\lib\site-packages\mpd\base.py", line 378, in command_callback
    res = self._wrap_iterator(res)
  File "C:\Python27\lib\site-packages\mpd\base.py", line 567, in _wrap_iterator
    return list(iterator)
  File "C:\Python27\lib\site-packages\mpd\base.py", line 286, in _parse_list
    for key, value in self._parse_pairs(lines):
  File "C:\Python27\lib\site-packages\mpd\base.py", line 226, in _parse_pairs
    for line in lines:
  File "C:\Python27\lib\site-packages\mpd\base.py", line 548, in _read_lines
    line = self._read_line()
  File "C:\Python27\lib\site-packages\mpd\base.py", line 528, in _read_line
    if not line.endswith("\n"):
AttributeError: 'NoneType' object has no attribute 'endswith'
Exception AttributeError: "'NoneType' object has no attribute 'path'" in <function _remove at 0x0000000004A9AF98> ignored

Process finished with exit code 1

    Any useful info for debug?
    Could you point me more? Thanks.

  13. Mic92 commented on Feb 28, 2018

    @Mic92
    Mic92
    on Feb 28, 2018
    Owner

    s is also a key in pdb, you did not print anything but just skipped execution. Use print(s) instead.

  14. handsomegui commented on Feb 28, 2018

    @handsomegui
    handsomegui
    on Feb 28, 2018
    Author

    @Mic92

    Sorry for my poor knowledge on this.

    Here is what i get:

    Artist: ?????��????ƽ?

--Return--
> c:\python27\lib\site-packages\mpd\base.py(53)decode_str()->None
-> import pdb; pdb.set_trace()
(Pdb)

  15. Mic92 commented on Feb 28, 2018

    @Mic92
    Mic92
    on Feb 28, 2018
    Owner

    Can you print the following?

    print([ c for c in s])

    And post the result also upstream.

  16. Mic92 commented on Feb 28, 2018

    @Mic92
    Mic92
    on Feb 28, 2018
    Owner

    Maybe also share the file with upstream.

  17. handsomegui commented on Feb 28, 2018

    @handsomegui
    handsomegui
    on Feb 28, 2018
    Author

    Thanks @Mic92

    Here is the output:

    Artist: ?????��????ƽ?

['A', 'r', 't', 'i', 's', 't', ':', ' ', '?', '?', '?', '?', '?', '\xc0', '\xa1', '?', '?', '?', '?', '\xc6', '\xbd', '?', '\n']
--Return--
> c:\python27\lib\site-packages\mpd\base.py(54)decode_str()->None
-> import pdb; pdb.set_trace()
(Pdb)

    Sorry. I still have no clue to find out the specific file :(

  18. handsomegui commented on Feb 28, 2018

    @handsomegui
    handsomegui
    on Feb 28, 2018
    Author

    Unfortunately, the upstream admin closed my issue report.

  19. Mic92 commented on Feb 28, 2018

    @Mic92
    Mic92
    on Feb 28, 2018
    Owner

    well, just install wireshark.

  20. guizhenwei commented on Mar 21, 2018

    @guizhenwei
    guizhenwei
    on Mar 21, 2018

    FYI.

    I finally picked out the file which caused the UTF-8 decoding error:

    https://www.dropbox.com/s/f72ti62v7op6biu/KPSU.mp3?dl=0

    I've reported this to MPD issues list also.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions