diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 2424eb2f..d11ff43a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -60,7 +60,7 @@ For other questions, discussions, and chats, we have: Bug reports =========== -When `reporting a bug`__ please include: +When reporting a bug please include: * Your operating system name, version and architecture (32 or 64 bits). * Your Python version. diff --git a/README.rst b/README.rst index be2be1c4..a47070bc 100644 --- a/README.rst +++ b/README.rst @@ -37,12 +37,12 @@ For more comprehensive installation instructions and development instructions, s For development instructions, see: `Development Instructions `_ -Make sure you have Python 2.7 installed: - * Download and install Python 2.7 32 bits for Windows - https://www.python.org/ftp/python/2.7.14/python-2.7.14.msi - * Download and install Python 2.7 for Mac - https://www.python.org/ftp/python/2.7.14/python-2.7.14-macosx10.6.pkg - * Download and install Python 2.7 for Linux via distro package manager +Make sure you have Python 3.6+ installed: + * Download and install Python 3.6+ Windows + https://www.python.org/downloads/windows/ + * Download and install Python 3.6+ for Mac + https://www.python.org/downloads/mac-osx/ + * Download and install Python 3.6+ for Linux via distro package manager Next, download and extract the latest DeltaCode release from:: diff --git a/docs/source/conf.py b/docs/source/conf.py index c5be8578..b519cddf 100644 --- a/docs/source/conf.py +++ b/docs/source/conf.py @@ -16,7 +16,6 @@ # -- Project information ----------------------------------------------------- - project = 'deltacode' copyright = 'nexb Inc.' author = 'nexb Inc.' @@ -49,7 +48,7 @@ # Add any paths that contain custom static files (such as style sheets) here, # relative to this directory. They are copied after the builtin static files, # so a file named "default.css" will overwrite the builtin "default.css". -html_static_path = ['_static'] +html_static_path = [] html_context = { "display_github": True, diff --git a/docs/source/deltacode/GSOC_2021_report.rst b/docs/source/deltacode/GSOC_2021_report.rst new file mode 100644 index 00000000..fa89f780 --- /dev/null +++ b/docs/source/deltacode/GSOC_2021_report.rst @@ -0,0 +1,67 @@ +Google Summer of Code 2021 - Final report +========================================= + +Project: Virtual Codebase support in DeltaCode +---------------------------------------------------------- + +**Pratik Dey ** + +Project Overview +---------------- + +The goal of this proposal is to refactor DeltaCode to use Scancode-Toolkit’s Virtual Codebase class. +This refactoring will allow DeltaCode to be a library as opposed to only be used as a CLI tool, moreover, +this refactor will allow DeltaCode to determine deltas much more effectively in the form of BFS tree scan +of the two tree structures unlike indexing the entire codebase. + + +Main Objectives of the project +------------------------------ + +- Migrate to using VirtualCodebase from the latest scancode. +- Create DeltaCode documentation on Read The Docs. +- Provide the support for fingerprint plugin for Virtual Codebase. +- Provide the Support for enabling Virtual Codebase to scan files having full root paths as their location. + +The Project +----------- + +- Virtual Codebase Integration with Deltacode +- Removing redundant File and License Objects +- Provided options in deltacode scans +- Added Docker Script for Dockerizing the Deltacode Application and make it platform-independent. +- Add Read the Docs Support to Deltacode. + +I have completed all the tasks that were in the scope of this GSoC project. + +Pull Requests +------------- + +- https://github.com/nexB/deltacode/pull/167 [Merged] +- https://github.com/nexB/deltacode/pull/176 [Open] +- https://github.com/nexB/deltacode/pull/171 [Open] +- https://github.com/nexB/deltacode/pull/178 [Open] +- https://github.com/nexB/deltacode/pull/168 [Open] + +Links +----- + +.. + [Project Link] https://summerofcode.withgoogle.com/organizations/5312205181943808/#6516503005888512 + +- `Project Details `_ +- `Proposal `_ +- `ScanCode Toolkit `_ +- `DeltaCode `_ + +------------ + +I’ve had a wonderful time during these three months and have learned plenty of things. I would +really like to thank `@pombredanne `_, +`@majurg `_, and `@JonoYang `_ for their +constant support throughout the journey. From good job claps to nit-picky constructive +code-reviews, I enjoyed every bit of this GSoC project. + +I had a wonderful time during the GSOC, I learned a lot of things during this time. I really enjoyed this project. +I would really like to thank my mentors `@pombredanne `_, `@majurg `_, and `@TG1999 `_, +and all other About code members who constantly supported me throughout this project. diff --git a/docs/source/deltacode/comprehensive_installation.rst b/docs/source/deltacode/comprehensive_installation.rst index e387a869..3cbadf6e 100644 --- a/docs/source/deltacode/comprehensive_installation.rst +++ b/docs/source/deltacode/comprehensive_installation.rst @@ -1,7 +1,7 @@ Comprehensive Installation ========================== -DeltaCode requires Python 2.7.x and is tested on Linux, Mac, and Windows. Make sure Python 2.7 is +DeltaCode requires Python 3.6+ and is tested on Linux, Mac, and Windows. Make sure Python 3.6 (or above) is installed first. System Requirements @@ -20,11 +20,11 @@ System Requirements Prerequisites ------------- -DeltaCode needs a Python 2.7 interpreter. +DeltaCode needs a Python 3.6(or above) interpreter. - **On Linux**: - Use your package manager to install ``python2.7``. If Python 2.7 is not available from your + Use your package manager to install atleast ``python3.6``. If Python 3.6 is not available from your package manager, you must compile it from sources. For instance, visit https://github.com/dejacode/about-code-tool/wiki/BuildingPython27OnCentos6 @@ -32,12 +32,8 @@ DeltaCode needs a Python 2.7 interpreter. - **On Windows**: - Use the Python 2.7 32-bit (e.g. the Windows x86 MSI installer) for X86 regardless of whether - you run Windows on 32-bit or 64-bit. **DO NOT USE Python X86_64 installer** even if you run - 64 bit Windows. - Download Python from this url: - https://www.python.org/ftp/python/2.7.14/python-2.7.14.msi + https://www.python.org/ Install Python on the c: drive and use all default installer options. See the Windows installation section for more installation details. @@ -46,7 +42,7 @@ DeltaCode needs a Python 2.7 interpreter. Download and install Python from this url: - https://www.python.org/ftp/python/2.7.14/python-2.7.14-macosx10.6.pkg + https://www.python.org/ Installation on Linux and Mac ----------------------------- @@ -123,3 +119,16 @@ Un-installation - Delete the directory in which you extracted DeltaCode. - Delete any temporary files created in your system temp directory under a deltacode directory. + + +Using the docker image for testing Deltacode +--------------- + +- In the project root directory run `docker-compose up`. +- This will create an image of deltacode with the name `delta_code`. +- To verify the image created run `docker image ls`. +- To run the image run `docker run -itd --name delta_code `. +- The above command runs the image in the background and creates a container with the name as per specified. +- To execute the container in a bash mode run `docker exec -it bash`. +- The above command will open a bash shell in the container. +- To run the commands / pytest inside the shell you can use the commands as specified in the documentations. diff --git a/docs/source/deltacode/index.rst b/docs/source/deltacode/index.rst index 3a742ab5..f35e641a 100644 --- a/docs/source/deltacode/index.rst +++ b/docs/source/deltacode/index.rst @@ -13,3 +13,4 @@ Welcome to the DeltaCode Documentation. development json_to_csv release_process + GSOC_2021_report diff --git a/docs/source/index.rst b/docs/source/index.rst index 1e58ac07..ff9f1c07 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -2,8 +2,14 @@ Welcome to deltacode's documentation! ========================================= .. toctree:: - :maxdepth: 2 - :caption: Contents: + :maxdepth: 1 deltacode/index +Indices and tables +================== + +* :ref:`genindex` +* :ref:`modindex` +* :ref:`search` +======= diff --git a/test.csv b/test.csv new file mode 100644 index 00000000..00dd1d66 --- /dev/null +++ b/test.csv @@ -0,0 +1,142 @@ +Status,Score,Factors,Path,Name,Type,Size,Old Path +added,100,,coalib/coala_modes.py,coala_modes.py,file,3383, +added,100,,coalib/bearlib/aspects/__init__.py,__init__.py,file,3402, +added,100,,coalib/bearlib/aspects/base.py,base.py,file,2106, +added,100,,coalib/bearlib/aspects/docs.py,docs.py,file,1522, +added,100,,coalib/bearlib/aspects/meta.py,meta.py,file,2121, +added,100,,coalib/bearlib/aspects/Metadata.py,Metadata.py,file,7125, +added,100,,coalib/bearlib/aspects/Redundancy.py,Redundancy.py,file,6576, +added,100,,coalib/bearlib/aspects/taste.py,taste.py,file,2896, +added,100,,coalib/bearlib/languages/Language.py,Language.py,file,14576, +added,100,,coalib/bearlib/languages/definitions/C.py,C.py,file,854, +added,100,,coalib/bearlib/languages/definitions/CPP.py,CPP.py,file,1580, +added,100,,coalib/bearlib/languages/definitions/CSharp.py,CSharp.py,file,261, +added,100,,coalib/bearlib/languages/definitions/CSS.py,CSS.py,file,304, +added,100,,coalib/bearlib/languages/definitions/Java.py,Java.py,file,325, +added,100,,coalib/bearlib/languages/definitions/JavaScript.py,JavaScript.py,file,372, +added,100,,coalib/bearlib/languages/definitions/Python.py,Python.py,file,414, +added,100,,coalib/bearlib/languages/definitions/Unknown.py,Unknown.py,file,91, +added,100,,coalib/bearlib/languages/definitions/Vala.py,Vala.py,file,345, +added,100,,coalib/misc/Compatibility.py,Compatibility.py,file,144, +added,100,,coalib/output/ConsoleInteraction.py,ConsoleInteraction.py,file,32748, +added,100,,coalib/output/Logging.py,Logging.py,file,2097, +added,100,,coalib/results/result_actions/IgnoreResultAction.py,IgnoreResultAction.py,file,4109, +added,100,,coalib/results/result_actions/PrintAspectAction.py,PrintAspectAction.py,file,704, +added,100,,coalib/settings/Annotations.py,Annotations.py,file,1580, +added,100,,coalib/testing/BearTestHelper.py,BearTestHelper.py,file,513, +added,100,,coalib/testing/LocalBearTestHelper.py,LocalBearTestHelper.py,file,9119, +modified,55,Similar with hamming distance : 35,coalib/parsing/LineParser.py,LineParser.py,file,6902, +modified,55,Similar with hamming distance : 35,coalib/processes/communication/LogMessage.py,LogMessage.py,file,1643, +modified,54,Similar with hamming distance : 34,coalib/coala.py,coala.py,file,3133, +modified,54,Similar with hamming distance : 34,coalib/output/JSONEncoder.py,JSONEncoder.py,file,1348, +modified,53,Similar with hamming distance : 33,coalib/bearlib/languages/documentation/DocumentationComment.py,DocumentationComment.py,file,8609, +modified,53,Similar with hamming distance : 33,coalib/output/printers/LogPrinter.py,LogPrinter.py,file,6096, +modified,53,Similar with hamming distance : 33,coalib/parsing/ConfParser.py,ConfParser.py,file,5180, +modified,52,Similar with hamming distance : 32,coalib/results/SourceRange.py,SourceRange.py,file,6248, +modified,52,Similar with hamming distance : 32,coalib/settings/DocstringMetadata.py,DocstringMetadata.py,file,2505, +modified,51,Similar with hamming distance : 31,coalib/coala_delete_orig.py,coala_delete_orig.py,file,1573, +modified,51,Similar with hamming distance : 31,coalib/coala_main.py,coala_main.py,file,5853, +modified,50,Similar with hamming distance : 30,coalib/bearlib/languages/documentation/DocstyleDefinition.py,DocstyleDefinition.py,file,7947, +modified,50,Similar with hamming distance : 30,coalib/results/Diff.py,Diff.py,file,18608, +modified,48,Similar with hamming distance : 28,coalib/results/result_actions/ApplyPatchAction.py,ApplyPatchAction.py,file,1931, +modified,47,Similar with hamming distance : 27,coalib/bears/Bear.py,Bear.py,file,14966, +modified,46,Similar with hamming distance : 26,coalib/bearlib/abstractions/ExternalBearWrap.py,ExternalBearWrap.py,file,7569, +modified,46,Similar with hamming distance : 26,coalib/bears/BEAR_KIND.py,BEAR_KIND.py,file,71, +modified,46,Similar with hamming distance : 26,coalib/results/TextRange.py,TextRange.py,file,4507, +modified,45,Similar with hamming distance : 25,coalib/results/LineDiff.py,LineDiff.py,file,2418, +modified,44,Similar with hamming distance : 24,coalib/bearlib/naming_conventions/__init__.py,__init__.py,file,3745, +modified,43,Similar with hamming distance : 23,coalib/bearlib/abstractions/Linter.py,Linter.py,file,34235, +modified,43,Similar with hamming distance : 23,coalib/collecting/Collectors.py,Collectors.py,file,12482, +modified,43,Similar with hamming distance : 23,coalib/misc/Exceptions.py,Exceptions.py,file,1059, +modified,43,Similar with hamming distance : 23,coalib/output/printers/ListLogPrinter.py,ListLogPrinter.py,file,978, +modified,43,Similar with hamming distance : 23,coalib/parsing/CliParsing.py,CliParsing.py,file,4729, +modified,42,Similar with hamming distance : 22,coalib/processes/Processing.py,Processing.py,file,30267, +modified,41,Similar with hamming distance : 21,coalib/bearlib/spacing/SpacingHelper.py,SpacingHelper.py,file,3825, +modified,39,Similar with hamming distance : 19,coalib/misc/Caching.py,Caching.py,file,5773, +modified,38,Similar with hamming distance : 18,coalib/results/HiddenResult.py,HiddenResult.py,file,639, +modified,38,Similar with hamming distance : 18,coalib/results/SourcePosition.py,SourcePosition.py,file,1282, +modified,37,Similar with hamming distance : 17,coalib/collecting/Importers.py,Importers.py,file,6342, +modified,35,Similar with hamming distance : 15,coalib/misc/CachingUtilities.py,CachingUtilities.py,file,7156, +modified,35,Similar with hamming distance : 15,coalib/misc/Shell.py,Shell.py,file,4848, +modified,35,Similar with hamming distance : 15,coalib/results/TextPosition.py,TextPosition.py,file,1081, +modified,35,Similar with hamming distance : 15,coalib/settings/ConfigurationGathering.py,ConfigurationGathering.py,file,14106, +modified,35,Similar with hamming distance : 15,coalib/settings/FunctionMetadata.py,FunctionMetadata.py,file,11704, +modified,34,Similar with hamming distance : 14,coalib/bears/LocalBear.py,LocalBear.py,file,1522, +modified,34,Similar with hamming distance : 14,coalib/collecting/Dependencies.py,Dependencies.py,file,1311, +modified,34,Similar with hamming distance : 14,coalib/misc/BuildManPage.py,BuildManPage.py,file,7292, +modified,34,Similar with hamming distance : 14,coalib/settings/SectionFilling.py,SectionFilling.py,file,3855, +modified,33,Similar with hamming distance : 13,coalib/bears/GlobalBear.py,GlobalBear.py,file,1175, +modified,33,Similar with hamming distance : 13,coalib/parsing/Globbing.py,Globbing.py,file,13234, +modified,32,Similar with hamming distance : 12,coalib/processes/BearRunning.py,BearRunning.py,file,24044, +modified,32,Similar with hamming distance : 12,coalib/results/result_actions/ResultAction.py,ResultAction.py,file,3516, +modified,30,Similar with hamming distance : 10,coalib/settings/Section.py,Section.py,file,8750, +modified,29,Similar with hamming distance : 9,coalib/bearlib/languages/documentation/doxygen.coalang,doxygen.coalang,file,1433, +modified,28,Similar with hamming distance : 8,coalib/settings/Setting.py,Setting.py,file,8415, +modified,27,Similar with hamming distance : 7,coalib/results/ResultFilter.py,ResultFilter.py,file,9630, +modified,26,Similar with hamming distance : 6,coalib/results/AbsolutePosition.py,AbsolutePosition.py,file,2119, +modified,25,Similar with hamming distance : 5,coalib/bearlib/abstractions/SectionCreatable.py,SectionCreatable.py,file,2611, +modified,25,Similar with hamming distance : 5,coalib/bearlib/languages/documentation/DocumentationExtraction.py,DocumentationExtraction.py,file,11010, +modified,25,Similar with hamming distance : 5,coalib/misc/DictUtilities.py,DictUtilities.py,file,1360, +modified,20,,coalib/__init__.py,__init__.py,file,587, +modified,20,,coalib/coala_ci.py,coala_ci.py,file,347, +modified,20,,coalib/coala_format.py,coala_format.py,file,333, +modified,20,,coalib/coala_json.py,coala_json.py,file,326, +modified,20,,coalib/VERSION,VERSION,file,7, +modified,20,,coalib/bearlib/__init__.py,__init__.py,file,5572, +modified,20,,coalib/bearlib/languages/__init__.py,__init__.py,file,482, +modified,20,,coalib/bearlib/languages/LanguageDefinition.py,LanguageDefinition.py,file,3575, +modified,20,,coalib/bearlib/languages/documentation/default.coalang,default.coalang,file,392, +modified,20,,coalib/output/ConfWriter.py,ConfWriter.py,file,3863, +modified,20,,coalib/output/Interactions.py,Interactions.py,file,1200, +modified,20,,coalib/output/printers/LOG_LEVEL.py,LOG_LEVEL.py,file,387, +modified,20,,coalib/parsing/DefaultArgParser.py,DefaultArgParser.py,file,8787, +modified,20,,coalib/processes/CONTROL_ELEMENT.py,CONTROL_ELEMENT.py,file,114, +modified,20,,coalib/results/Result.py,Result.py,file,11406, +modified,20,,coalib/results/RESULT_SEVERITY.py,RESULT_SEVERITY.py,file,335, +modified,20,,coalib/results/result_actions/OpenEditorAction.py,OpenEditorAction.py,file,6600, +modified,20,,coalib/results/result_actions/PrintDebugMessageAction.py,PrintDebugMessageAction.py,file,611, +modified,20,,coalib/results/result_actions/PrintMoreInfoAction.py,PrintMoreInfoAction.py,file,617, +modified,20,,coalib/results/result_actions/ShowPatchAction.py,ShowPatchAction.py,file,5296, +unmodified,0,,coalib/default_coafile,default_coafile,file,0, +unmodified,0,,coalib/bearlib/abstractions/__init__.py,__init__.py,file,110, +unmodified,0,,coalib/bearlib/languages/definitions/__init__.py,__init__.py,file,341, +unmodified,0,,coalib/bearlib/languages/documentation/__init__.py,__init__.py,file,131, +unmodified,0,,coalib/bearlib/spacing/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/bears/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/collecting/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/misc/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/misc/Enum.py,Enum.py,file,270, +unmodified,0,,coalib/output/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/output/printers/__init__.py,__init__.py,file,269, +unmodified,0,,coalib/parsing/__init__.py,__init__.py,file,167, +unmodified,0,,coalib/processes/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/processes/LogPrinterThread.py,LogPrinterThread.py,file,688, +unmodified,0,,coalib/processes/communication/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/results/__init__.py,__init__.py,file,0, +unmodified,0,,coalib/results/result_actions/__init__.py,__init__.py,file,145, +unmodified,0,,coalib/settings/__init__.py,__init__.py,file,0, +removed,0,,coalib/coala_dbus.py,coala_dbus.py,file,1515, +removed,0,,coalib/bearlib/abstractions/Lint.py,Lint.py,file,15710, +removed,0,,coalib/bearlib/languages/definitions/c.coalang,c.coalang,file,568, +removed,0,,coalib/bearlib/languages/definitions/cpp.coalang,cpp.coalang,file,1057, +removed,0,,coalib/bearlib/languages/definitions/python3.coalang,python3.coalang,file,197, +removed,0,,coalib/bears/requirements/GemRequirement.py,GemRequirement.py,file,1067, +removed,0,,coalib/bears/requirements/NpmRequirement.py,NpmRequirement.py,file,841, +removed,0,,coalib/bears/requirements/PackageRequirement.py,PackageRequirement.py,file,4261, +removed,0,,coalib/bears/requirements/PipRequirement.py,PipRequirement.py,file,847, +removed,0,,coalib/misc/Annotations.py,Annotations.py,file,1580, +removed,0,,coalib/misc/ContextManagers.py,ContextManagers.py,file,7320, +removed,0,,coalib/misc/Future.py,Future.py,file,3748, +removed,0,,coalib/misc/MutableValue.py,MutableValue.py,file,80, +removed,0,,coalib/misc/StringConverter.py,StringConverter.py,file,5054, +removed,0,,coalib/output/dbus/__init__.py,__init__.py,file,561, +removed,0,,coalib/output/dbus/BuildDbusService.py,BuildDbusService.py,file,1411, +removed,0,,coalib/output/dbus/DbusApp.py,DbusApp.py,file,1509, +removed,0,,coalib/output/dbus/DbusDocument.py,DbusDocument.py,file,6938, +removed,0,,coalib/output/dbus/DbusServer.py,DbusServer.py,file,5769, +removed,0,,coalib/parsing/StringProcessing/__init__.py,__init__.py,file,1082, +removed,0,,coalib/parsing/StringProcessing/Core.py,Core.py,file,21420, +removed,0,,coalib/parsing/StringProcessing/Filters.py,Filters.py,file,1331, +removed,0,,coalib/parsing/StringProcessing/InBetweenMatch.py,InBetweenMatch.py,file,2120, +removed,0,,coalib/parsing/StringProcessing/Match.py,Match.py,file,1541, +moved,0,,coalib/testing/__init__.py,__init__.py,file,0,coalib/bears/requirements/__init__.py