Language stats inconsistent between File Info Dashboard and filtered Table View

[mjherzog]

I am trying to assemble Development Codebase Statistics by Programming Language. The results from the Programming Languages pie-chart on the File Info Dashboard do not match the results from Table View. The best example is for C++.

1. For the Languages pie-chart I included "No Value Detected" and get 64.5% for C++ which is 4,167 files based on the total of 6461 files.
2. For the Table View, I first filtered on Files which returns 6461 (same as Dashboard). The Programming Language filter for C++ returns 4,798 files.

There are similar discrepancies with the No Value Detected data.

System configuration

For bug reports, it really helps us to know:

• What OS are you running on? Windows 10
• What version of scancode-workbench are you running on? version 3.10
• What version of scancode-toolkit was used to generate the scan file? version 3.0.2

[steven-esser]

Will look into this.

[steven-esser]

@mjherzog The culprit in this case is the LEGEND_LIMIT variable here: https://github.com/nexB/scancode-workbench/blob/d423c41f74787f77a37ab27363899ac12eead7a9/assets/app/js/controllers/fileDashboard.js#L32

This variable controls the number of pie chart slices that show up in the chart itself, and the percentages are calculated w.r.t. the number of items visible on the pie chart during draw time. This is why we are seeing a discrepancy with the ScanData View. Increasing the number to something larger or an infinite amount (necessary to get proper percentage values) unfortunately causes the chart to be illegible when there are many values being displayed.

We may need a need way of displaying these stats, since they are critical to a lot of our work. One solution could be the addition of a table below the pie charts that displays the proper data, with totals and percentages calculated correctly.

That is just one option. If we don't want to directly display this information, we could have a reporting feature of sorts that spits out csv file with that data.

@mjherzog Feedback appreciated.

[mjherzog]

A display table below with the number of files per Language would be a good next step - we could cut limit the display as (1) single number threshold - e.g. only show in table if 50 files or (2) percentage threshold - e.g. only show Languages where number of files is 5% or so of total files. Either would give us most of the data we need for audit planning and we can always check the base Table View for the odds & ends.

[steven-esser]

@mjherzog Sounds good to me 👍

[mjherzog]

The cleanest solution would be to provide a table below each pie-chart with the number for each element that appears in the pie-chart.

[mjherzog]

adding this to 3.2.0 in the hope that we can do something pretty quickly - the core point is to show the numbers in tandem with the existing % based charts. We have "room" on each of the current Dashboard pages for a table or chart below each of the existing charts.

[steven-esser]

@mjherzog This is a common use case and I am all for including it in v3.2.0

[OmkarPh]

we have % and actual count available in chart tooltip as discussed in #581
Can we close this issue ?
CC @mjherzog @AyanSinhaMahapatra @pombredanne @steven-esser