Skip to content

Commit 66f5995

Browse files
committed
Refine README format
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
1 parent 3dd14d7 commit 66f5995

1 file changed

Lines changed: 52 additions & 41 deletions

File tree

README.rst

Lines changed: 52 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -64,20 +64,22 @@ Analyzing a traced build is a multi-stage process that involves:
6464
build,
6565

6666
- analyzing the build graph to determine the source to target relationships,
67-
such as source code files being built into a binary,
67+
such as source code files being built into a binary,
6868

6969
- optionally creating graphical representations to visualize subset of
7070
your build graph.
7171

72-
Each of these steps is performed by invoking tracecode from the command line
72+
Each of these steps is performed by invoking `tracecode` from the command line
7373
with different options and arguments.
7474

7575

7676
Run the trace analysis with::
77+
7778
tracecode <options> <command> <arguments>
7879

7980

8081
For command help use::
82+
8183
tracecode -h
8284

8385

@@ -91,6 +93,7 @@ Tutorial
9193

9294
1. Parse the collected raw traces
9395
---------------------------------
96+
9497
Create a new empty directory to store parsed traces. Then parse using the
9598
"parse" command::
9699
@@ -102,7 +105,8 @@ This will parse the traces and ensure they can be processed and are complete
102105

103106
2. Collect the inventory of files processed during the tracing
104107
--------------------------------------------------------------
105-
If traces are complete the next step is to collect the inventories of files
108+
109+
If traces are consistent the next step is to collect the inventories of files
106110
reads and writes. Use the "list" command (which should be called inventory).
107111
It creates two files from a parsed trace: a list of files being only read and
108112
a list of files being written::
@@ -115,22 +119,24 @@ The list command extracts all the paths used in the traces.
115119

116120
3. optional but recommended: Filter your parsed traces
117121
------------------------------------------------------
122+
118123
The next step is to review these reads and writes and decide which ones could
119124
be filtered out as they may not contribute interesting data to the build graph
120125
and the analysis.
121126

122127
This includes typically:
123-
/etc/*
124-
/proc/*
125-
the build log files if any
126-
Some standard things in /usr/* and similar
128+
129+
- /etc/*
130+
- /proc/*
131+
- the build log files if any
132+
- Some standard things in /usr/* and similar
127133

128134
For this you build a list of reads to ignore and writes to ignore (usually
129135
patterns or plain lists) you stuffs these two lists in a two files and use the
130-
filter command to filter out these reads and writes
136+
filter command to filter out these reads and writes.
131137

132138
Beware of not filtering too much: temp files in /tmp you want to keep certain
133-
makedepend (.po, etc) files you may not care for
139+
makedepend (.po, etc) files you may not care for.
134140

135141
When you filter at first filter to a new directory so taht you do not replace
136142
the original full parsed traces yet, so you can get comfy and refine your
@@ -141,23 +147,25 @@ out or prune from the trace Either a full path as found in the reads or writes
141147
list, or a pattern as in /etc/* in which case everything matching /etc/* would
142148
be filtered out like when you use glob patterns on the command line Use oe
143149
path or pattern per line in a file. Note that it can be a single column csv
144-
alright too
150+
alright too.
145151

146152

147153
4. optional: Guess sources and targets
148154
-------------------------------------
155+
149156
You can use the "guess" command to guess sources and targets, but that is just
150157
a guess. Guessing works ok on small well defined simple codebases, but might
151-
noy likely be good on larger ones
158+
noy likely be good on larger ones.
152159

153160
The guess goes this way:
154-
* files that are only ever read from are likely the source/devel
155-
* files that are only ever written to read are likely the target/deployed
161+
- files that are only ever read from are likely the source/devel
162+
- files that are only ever written to read are likely the target/deployed
156163

157164

158165

159166
5. Assemble the inventory of sources an targets
160167
-----------------------------------------------
168+
161169
Once you have filtered your parsed trace, you need to create a list of files
162170
that are your sources, origin development files and another list that are your
163171
targets, deployed files. You need to build theses inventories each in a
@@ -174,12 +182,14 @@ devel/sources files and what is the list of deployed/targets files.
174182

175183
6. Analyze sources to targets transformations
176184
---------------------------------------------
185+
177186
Then you can run either the analyze command to get the source to target
178187
deployment analysis.
179188

180189

181190
7. optional: Graph select subset of sources to targets transformations
182191
----------------------------------------------------------------------
192+
183193
You can selectively create a graphic tracing the transformation from several
184194
sources to a one target or several targets to one sources with graphics
185195
(selectively because this takes long time to run and large graphics are
@@ -190,8 +200,9 @@ impossible to visualize)
190200
FAQ:
191201
----
192202

193-
Q: When parsing raw traces I am getting this error:
194-
ERROR:tracecode:INCOMPLETE TRACE, 149249 orphaned trace(s) detected. First pid is: 3145728.
203+
Q: When parsing raw traces I am getting this error::
204+
205+
ERROR:tracecode:INCOMPLETE TRACE, 149249 orphaned trace(s) detected. First pid is: 3145728.
195206

196207
A: This is a serious error and means that your trace is not coherent as some
197208
process traces could not be related to the initial command launch graph and
@@ -200,8 +211,9 @@ commands and store the strace output in the same directory. You need to
200211
recollect your traces starting with a clean empty directory.
201212

202213

203-
Q: When parsing raw traces I am getting several warnings:
204-
WARNING:tracecode:parse_line: Unable to decode descriptor for pid: 3097012, line: '1399882436.807573 dup2(5</extra/linux-2.6.32/scripts/mksysmap>, 255) = 255\n'
214+
Q: When parsing raw traces I am getting several warnings::
215+
216+
WARNING:tracecode:parse_line: Unable to decode descriptor for pid: 3097012, line: '1399882436.807573 dup2(5</extra/linux-2.6.32/scripts/mksysmap>, 255) = 255\n'
205217

206218
A: This is just a warning that you can ignore most of the times. Here a file
207219
descriptor 255 does not (and cannot) exist, hence the warning.
@@ -210,41 +222,40 @@ descriptor 255 does not (and cannot) exist, hence the warning.
210222
Credits and related tools
211223
-------------------------
212224

213-
This implementation of an strace-based build tracer is
214-
essentially an implementation of this papers:
225+
This implementation of an strace-based build tracer is essentially an implementation
226+
of this papers:
215227

216-
Sander van der Burg published this article:
217-
http://sandervanderburg.blogspot.be/2012/04/dynamic-analysis-of-build-processes-to.html
228+
Sander van der Burg published an article and paper:
218229

219-
... about this paper:
220-
http://www.st.ewi.tudelft.nl/~sander/index.php/publications
230+
- http://sandervanderburg.blogspot.be/2012/04/dynamic-analysis-of-build-processes-to.html
231+
- "Discovering Software License Constraints: Identifying a Binary's Sources by Tracing Build Processes"
232+
- http://www.st.ewi.tudelft.nl/~sander/pdf/publications/TUD-SERG-2012-010.pdf
233+
- By Sander van der Burg, Julius Davies, Eelco Dolstra, Daniel M. German, Armijn Hemel.
234+
Technical Report TUD-SERG-2012-010, Software Engineering Research Group, Delft, The Netherlands, April 2012.
221235

222-
"Discovering Software License Constraints:
223-
Identifying a Binary's Sources by Tracing Build Processes"
224-
By Sander van der Burg, Julius Davies, Eelco Dolstra,
225-
Daniel M. German, Armijn Hemel.
226-
Technical Report TUD-SERG-2012-010, Software Engineering Research Group, Delft, The Netherlands, April 2012.
227-
http://www.st.ewi.tudelft.nl/~sander/pdf/publications/TUD-SERG-2012-010.pdf
228236

229-
Later this similar paper relates a similar approach too:
230-
http://web.archive.org/web/20160329060541/http://shanemcintosh.org/assets/ase2014_vanderburg.pdf
231-
"Tracing Software Build Processes to Uncover License Compliance Inconsistencies"
232-
By Sander van der Burg, Eelco Dolstra, Shane McIntosh, Julius Davies
233-
Daniel M. German, and Armijn Hemel
237+
Later, this similar paper relates the same approach:
238+
239+
- "Tracing Software Build Processes to Uncover License Compliance Inconsistencies"
240+
- http://web.archive.org/web/20160329060541/http://shanemcintosh.org/assets/ase2014_vanderburg.pdf
241+
- By Sander van der Burg, Eelco Dolstra, Shane McIntosh, Julius Davies, Daniel M. German, and Armijn Hemel
234242

235243

236244
The Chromium test team built "swarming.client", a test isolation
237245
tools that was also a big inspiration for this tool too:
238-
https://www.chromium.org/developers/testing/isolated-testing/infrastructure
239-
https://chromium.googlesource.com/external/swarming.client/
240246

247+
- https://www.chromium.org/developers/testing/isolated-testing/infrastructure
248+
- https://chromium.googlesource.com/external/swarming.client/
249+
250+
251+
- memoize.py and fabricate use strace to track file dependencies
252+
using a similar approach to this tool.
253+
- https://github.com/kgaughan/memoize.py
254+
- https://code.google.com/archive/p/fabricate/
241255

242-
memoize.py and fabricate use strace to track file dependencies
243-
using a similar approach to this tool. And this article provides some good background on that topic:
256+
- And this article provides some good background on the same topic:
244257
https://news.ycombinator.com/item?id=9356433 :
245258

246-
- https://github.com/kgaughan/memoize.py
247-
- https://code.google.com/archive/p/fabricate/
248259

249260

250261
http://buildaudit.sourceforge.net/ is a related build tracing tool that
@@ -253,7 +264,7 @@ handles ptrace directly ass opposed to rely on strace for tracing.
253264

254265
Electric cloud is tool has some ways to track which files are accessed during a build
255266
using ptrace of LD_PRELOAD (or a custom file system)
256-
- https://electric-cloud.com/
267+
- https://electric-cloud.com/
257268

258269

259270
License

0 commit comments

Comments
 (0)