Skip to content

Bug/memory bloat fix - #4

Merged
ShivamYuvraj merged 10 commits into
masterfrom
bug/memory_bloat_fix
Feb 10, 2021
Merged

ShivamYuvraj merged 10 commits into
masterfrom
bug/memory_bloat_fix

Conversation

@ShivamYuvraj

Copy link
Copy Markdown

No description provided.

@asldevi asldevi left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

overall it looks good @ShivamYuvraj.

we should add at least two testcases at php/c level, whichever is easier.

  1. to check that everything is sane with less than DEFAULT_SPAN_LIMIT number of spans
  2. to check that everything is sane with >= DEFAULT_SPAN_LIMIT number of spans

Comment thread src/Trace/Integrations/Curl.php Outdated
* Static method to add tracer
*/
public static function setTracer($tracer){
PDO::$tracer = $tracer;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo here - replace it with Curl

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hv made some change in this logic

Comment thread ext/opencensus_trace.c Outdated
static void (*opencensus_original_zend_execute_ex) (zend_execute_data *execute_data);
static void (*opencensus_original_zend_execute_internal) (zend_execute_data *execute_data, zval *return_value);
// global value for max number of spans in any trace
static int SPAN_LIMIT = 100;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't we making this configurable from php side?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, not this actually though

Comment thread ext/opencensus_trace.c Outdated
}

// release zend string
zend_string_release(k);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is releasing k whereas this is releasing the key/value from hashmap OPENCENSUS_G(spans). am i right?

could you add these details as comments preceding those lines?

}

function opencensus_trace_remove_span($key): bool {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

any reason to add it here in daemon functions?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't affect anything adding here, but it lets developer know what are the c functions defined and their use

Comment thread src/Trace/Integrations/Curl.php Outdated
Comment on lines +70 to +74
// checks if spanlimit has reached and if yes flushes the closed spans
if (Curl::$tracer != null) {
Curl::$tracer->checkSpanLimit();
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this can't be expected out of every handler function and we should think of a better way for it.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Comment thread src/Trace/Tracer/ExtensionTracer.php Outdated
}
}

$this->export($closedSpans);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we rename this to capture removal part of it too?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@asldevi

asldevi commented Jan 27, 2021

Copy link
Copy Markdown

Also, add our approach of reducing memory bloat in the readme here, removing this part that I added before.

Comment thread ext/opencensus.c
PHP_FE(opencensus_trace_function, arginfo_opencensus_trace_function)
PHP_FE(opencensus_trace_method, arginfo_opencensus_trace_method)
PHP_FE(opencensus_trace_list, NULL)
PHP_FE(opencensus_trace_count, NULL)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

opencensus_span_count is a better name, but I think you kept it for being consistent with other names. let's leave it.

Comment thread ext/opencensus_trace.c
}

// deleting th span assosciated with the given span_id
if (zend_hash_del(OPENCENSUS_G(spans), k) != SUCCESS) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this deletes the corresponding entry of the span in the hashtable. does it remove the actual span too to be able to release memory actually? I mean how/where span_dtor is getting called from here?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, span_dtor is getting called from here, checked with log and also verified locally by using echo(get_human_readable_size(memory_get_peak_usage(true)));

@ShivamYuvraj ShivamYuvraj Feb 1, 2021 •

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When inserting the sapn, we also pass the destructor function,
example -> zend_hash_init(OPENCENSUS_G(spans), 16, NULL, span_dtor, 0);
this function gets called on zend_hash_del

https://www.phpinternalsbook.com/php5/hashtables/hashtable_api.html

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got it. thanks.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's put this info in comments

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Comment thread src/Trace/Tracer/ExtensionTracer.php Outdated
{
$count = opencensus_trace_count();

if ($count > $this->spanLimit) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: this should be >=

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Comment thread src/Trace/Tracer/ExtensionTracer.php Outdated
}

/* This checks the numbet of spans in memory and if the count is more than the set limit, it exports all
the closed span present in memory, to free up the memory */

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also add why we're exporting and deleting only closed spans.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done


/* This checks the numbet of spans in memory and if the count is more than the set limit, it exports all
the closed span present in memory, to free up the memory */
public function checkSpanLimit()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's rename SpanLimit as spanBufferLimit/spanBufferSize or something across the code so that it doesn't mislead as if we're limiting number of spans in a trace.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

// set span limit from options if present
if (isset($options['span_limit'])){
$this->spanLimit = $options['span_limit'];
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

*
* @param SpanContext|null $initialContext The starting span context.
* @param null $exporter.
* @param array $options

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add about the span_limit/span_buffer_size here

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@ShivamYuvraj
ShivamYuvraj merged commit 1968b08 into master Feb 10, 2021
@akashsharma95
akashsharma95 deleted the bug/memory_bloat_fix branch April 15, 2021 07:56
@akashsharma95
akashsharma95 restored the bug/memory_bloat_fix branch April 15, 2021 07:56
@akashsharma95
akashsharma95 deleted the bug/memory_bloat_fix branch April 15, 2021 07:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants