Skip to content

Improve array decoding performance - #32

Merged
gfx merged 1 commit into
msgpack:masterfrom
sergeyzenchenko:master
May 28, 2019
Merged

gfx merged 1 commit into
msgpack:masterfrom
sergeyzenchenko:master

Conversation

@sergeyzenchenko

Copy link
Copy Markdown
Collaborator

Hi @gfx I've used this msgpack library in my project and found small optimization that improves decoding speed by 3-4%.

@sergeyzenchenko

Copy link
Copy Markdown
Collaborator Author

For large arrays performance difference can be even bigger and also it will produce less trash for GC

@codecov-io

Copy link
Copy Markdown

Codecov Report

Merging #32 into master will increase coverage by <.01%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master      #32      +/-   ##
==========================================
+ Coverage   96.14%   96.14%   +<.01%     
==========================================
  Files          14       14              
  Lines         752      753       +1     
  Branches      158      158              
==========================================
+ Hits          723      724       +1     
  Misses         12       12              
  Partials       17       17
Impacted Files Coverage Δ
src/Decoder.ts 99.27% <100%> (ø) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c022de5...3d50366. Read the comment docs.

@gfx

gfx commented May 28, 2019

Copy link
Copy Markdown
Member

Tested with this benchmark script:

/* eslint-disable no-console */
import { encode, decode } from "../src";
// @ts-ignore
import _ from "lodash";

const data = _.cloneDeep(new Array(1000).fill([[1], [1, 2, 3], [1, 2, 3, 4, 5]]));

// warm up
const encoded = encode(data);
decode(encoded);

// run

console.time("encode");
for (let i = 0; i < 10000; i++) {
  encode(data);
}
console.timeEnd("encode");

console.time("decode");
for (let i = 0; i < 10000; i++) {
  decode(encoded);
}
console.timeEnd("decode");

And this PR significantly improves decoding:

before:

$ npx ts-node benchmark/sandbox.ts
encode: 2753.986ms
decode: 1855.263ms

after:

$ npx ts-node benchmark/sandbox.ts
encode: 2393.308ms
decode: 3804.347ms

Great!

@gfx

gfx commented May 28, 2019

Copy link
Copy Markdown
Member

(CI failed because of SauceLabs credentials; will fix it later https://docs.travis-ci.com/user/environment-variables/ )

@gfx
gfx merged commit 2589c87 into msgpack:master May 28, 2019
@nelsonlaquet

Copy link
Copy Markdown

I wonder if you could improve the performance further on certain engines by allocating an array without holes in it:

array: Array.from({ length: size }),

Source:
https://v8.dev/blog/elements-kinds
https://2ality.com/2018/12/creating-arrays.html

@gfx

gfx commented Jul 3, 2019 •

Copy link
Copy Markdown
Member

Tried it, but Array.from({ length: size }) makes decoding 10x slower than new Array(size) :-(

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants