Skip to content

[server and core] waveform returns lists min and max with zeros or bands of zeros #167

Closed
Closed
[server and core] waveform returns lists min and max with zeros or bands of zeros#167
Assignees
yomguypiemTointoin
Labels
bug

Description

@Tointoin
Tointoin
opened on Mar 25, 2020

/timeside/api/items/07836d4f-72d0-4a66-a076-f6f06cce572c/waveform/
return a waveform with min values all at -4.203895392974451e-45 and max all at 0.0.

Same case for many items recently generated (selection tests_DALI_yt).

Activity

  1. Tointoin
    added
    bug
    on Mar 25, 2020
  2. Tointoin
    assigned
    Tointoin
    and
    yomguy
    on Mar 30, 2020

  3. Tointoin commented on Mar 31, 2020

    @Tointoin
    Tointoin
    on Mar 31, 2020
    CollaboratorAuthor

    With a few log on waveform serializer code:

    result_id = 'waveform_analyzer'
        wav_res = h5py.File(result.hdf5.path, 'r').get(result_id)

        duration = wav_res['audio_metadata'].attrs['duration']
        server_logger.info(f'duration: {duration}') # LOGGING DURATION
        samplerate = wav_res['data_object'][
            'frame_metadata'].attrs['samplerate']
        server_logger.info(f'samplerate: {samplerate}') # LOGGING SAMPELRATE

        ...

        sample_values = np.round(self.time_values * samplerate).astype('int')

        for i in range(self.nb_pixels):
            values = wav_res['data_object']['value'][
                sample_values[i]:sample_values[i + 1]]
            if i < 10:                           # LOGGING 10 FIRST 
                server_logger.info(str(values))  # SAMPLE VALUE
            if values.size:
                self.min_values[i] = np.min(values)
                self.max_values[i] = np.max(values)

    I get quite normal log on a deezer preview 30 sec mp3 file:

    INFO 2020-03-31 19:11:15,232 serializers duration: 30.72
INFO 2020-03-31 19:11:15,233 serializers samplerate: 44100
INFO 2020-03-31 19:11:15,236 serializers [[ 0.15071355  0.05970893]
 [ 0.15789996  0.05962876]
 [ 0.16633978  0.05551996]
 ...
 [ 0.07510374 -0.04883934]
 [ 0.08088862 -0.04893086]
 [ 0.08825559 -0.04394966]]
INFO 2020-03-31 19:11:15,238 serializers [[ 0.09382088 -0.04255066]
 [ 0.09762768 -0.04794458]
 [ 0.10366786 -0.05218805]
 ...
 [ 0.05083828  0.04840193]
 [ 0.04840099  0.03691421]
 [ 0.04771852  0.02911852]]
INFO 2020-03-31 19:11:15,239 serializers [[0.04112494 0.0286877 ]
 [0.02681965 0.031142  ]
 [0.02052411 0.03037054]
 ...
 [0.         0.        ]
 [0.         0.        ]
 [0.         0.        ]]
INFO 2020-03-31 19:11:15,240 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:11:15,241 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:11:15,242 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:11:15,243 serializers [[0.         0.        ]
 [0.         0.        ]
 [0.         0.        ]
 ...
 [0.04067133 0.06245746]
 [0.0208528  0.06156785]
 [0.00379307 0.06160603]]
INFO 2020-03-31 19:11:15,244 serializers [[-0.01233356  0.05503304]
 [-0.02923564  0.05084599]
 [-0.03813753  0.06220904]
 ...
 [ 0.2581892   0.04209551]
 [ 0.25444093  0.04515055]
 [ 0.2494465   0.05317261]]
INFO 2020-03-31 19:11:15,245 serializers [[0.23985252 0.04020593]
 [0.23169126 0.02527791]
 [0.22214812 0.02236784]
 ...
 [0.         0.        ]
 [0.         0.        ]
 [0.         0.        ]]
INFO 2020-03-31 19:11:15,246 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]

    but empty sample value on a 270 sec youtube .m4a file:

    INFO 2020-03-31 19:22:53,549 serializers duration: 270.7219954648526
INFO 2020-03-31 19:22:53,550 serializers samplerate: 44100
INFO 2020-03-31 19:22:53,552 serializers [[ 0.  0.]
 [-0. -0.]
 [ 0.  0.]
 ...
 [ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]]
INFO 2020-03-31 19:22:53,553 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:22:53,555 serializers [[ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]
 ...
 [ 0.  0.]
 [-0. -0.]
 [ 0.  0.]]
INFO 2020-03-31 19:22:53,556 serializers [[-0. -0.]
 [ 0.  0.]
 [-0. -0.]
 ...
 [ 0.  0.]
 [ 0.  0.]
 [ 0.  0.]]
INFO 2020-03-31 19:22:53,557 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:22:53,558 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:22:53,560 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:22:53,561 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:22:53,563 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]
INFO 2020-03-31 19:22:53,565 serializers [[0. 0.]
 [0. 0.]
 [0. 0.]
 ...
 [0. 0.]
 [0. 0.]
 [0. 0.]]

    And same empty values on .opus from other item from youtube (also around 4 / 6 minutes long).

    Audio files are ok.

    @yomguy , @piem
    Does Aubio decode any format?

  4. piem commented on Mar 31, 2020

    @piem
    piem
    on Mar 31, 2020
    Member

    Does Aubio decode any format?

    yes, well, any format supported by libav when aubio is built with it.

    also, an exception should be raised if reading the file failed.

  5. Tointoin
    changed the title [-][server and core] waveform returns lists min and max with a constant value[/-] [+][server and core] waveform returns lists min and max with zeros or bands of zeros[/+] on Apr 1, 2020

  6. Tointoin commented on Apr 1, 2020

    @Tointoin
    Tointoin
    on Apr 1, 2020
    CollaboratorAuthor

    New problem detected with items that seemed to be ok:
    player example on a sweep
    player example on a 30 sec Deezer Preview
    It returns frames of zeros and frames with normal waveform values

  7. Tointoin commented on Apr 2, 2020

    @Tointoin
    Tointoin
    on Apr 2, 2020
    CollaboratorAuthor

    These bug seems to be provoked by use of Aubio decoder on server as default: I tried before and after commit 0286c7f.

    @yomguy, do we use gst decoder on sandbox for @gnuletik's player development until we fix it?

  8. Tointoin
    assigned
    piem
    on Apr 2, 2020

  9. yomguy commented on Apr 2, 2020

    @yomguy
    yomguy
    on Apr 2, 2020
    Member

    These bug seems to be provoked by use of Aubio decoder on server as default: I tried before and after commit 0286c7f.

    OK, could you please provided the output of the Result data on the same Item using each decoder so that we could compare?

    @yomguy, do we use gst decoder on sandbox for @gnuletik's player development until we fix it?

    Yes, let's switch back and see what happens..

  10. gnuletik
    closed this as completedon May 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions