Start up lag

I'm not sure if this is still an issue, but there have been a few reports in the 2.x series of pyfa taking a long time to start up. I'm creating this issue to link the other issues.

If you're experiencing this issue

First, please subscribe to the thread to receive new updates and posts! The subscribe button it to the right.

Attaching log files can help determine where the slow down is taking place. Your logs are found in %userprofile%/.pyfa and end in .log. Find the one that shows longish start up times (each log is time-stamped so it should be easy-ish to find gaps when it starts...). it may be helpful to dump the current log to disk by going to Preferences > Logging > Dump all logs

As I investigate, I may also post in this thread new builds that have additional logging to try to narrow down where the issues lie. it would be helpful to keep an eye on notification for any updates to this thread and provide logs and feedback as I work through this progress.

Thanks!

My pyfa has been starting noticeably slower since 2.0. Here are my logs
pyfa-2018-05-31.log

@Calavoow

Thanks @Calavoow! That log shows about 3 seconds of start up, which isn't too bad and kind of normal. Dump another log if you notice it taking much longer than that :) (unless of course this one as a long startup, and the log just didn't catch it?)

Another one, after a fresh start pyfa-2018-06-01.log
Seems like 13 seconds this time

Thanks!

I need to put a lot more debugging statements when various thing initialize to try to track down where this is taking a while. I'll try to get you a test build that you can run that should show a lot more granularity

pyfa_debug-2018-06-01.log

here's one of mine, I've been struggling with long startup times for a while

I'm attempting something that is either completely clever, or completely crazy

With bb978c3 you should now be able to specify a path to save profiling information. While this introduces additional overhead, as well as possible increased start up times, it will track the function calls within pyfa and allow me to determine what is eating up time. This will save to file in whatever directory you specify, and then you can upload it here for me to analyse.

If you don't specify a profile location, then it works just as it does now, so no overhead

I'll try to get a build out tonight and distribute in this thread for you guys to try this. I'm interested in seeing if it works out. :)

greetings all!

If you wouldn't mind, please download this build: https://ufile.io/dpw4a

You should be able to run this with the -p flag: open up command prompt, and run the pyfa.exe like so:

pyfa.exe -p c:\my\path\to\save\profiles

When you do this, pyfa will automatically save a profiled instance to the path you specify, in a file called pyfa-<data>.profile.

What I want you guys to do: start pyfa with that command, and when it loads, immediately close it. I don't want you to do a full on fitting session as that would probably be too much data to parse through. Just open it, wait for it to load, and then close it (or, if you want, go and dump the log as well). If you're able to get a lengthy start up, please upload the .profile file here and I can take a look at it.

Thanks!

@blitzmann

+1 on this. The development build of 2.x was starting slowly, but I figured that was before it went through the optimizations prior to being published. I was only half right. The published version is faster than the dev version, but still slower than 1.x Pyfa by a wide margin.

@blitzmann your ufile.io download is not working. Neither of the download buttons do anything.

your ufile.io download is not working. Neither of the download buttons do anything.

Works for me...

Here's an alternative: https://drive.google.com/file/d/14jrMCMd0wiN37VSobckJVM64Fz7CuIOL/view?usp=sharing

Download icon on top right corner

pyfa-20180604_180258.zip

Thanks! Will take a look. Can you also post the log file? What kind of delay were you getting?

I was getting 4-5 second startup time on my work laptop, which is i7 8650U with a fast NVMe SSD. It's slower (7-10 sec) on my gaming desktop at home. I can't get the log now, but will do so later.

I'm getting huge startup lag as well, it usually takes a couple of minutes for pyfa to start. Log attached.
pyfa-2018-06-18.log

@Leandrid

@Leandrid can you try #1605 (comment) (file mirror here #1605 (comment)) and post the profile? That is a huge startup time, and unfortunately the log only shows this

[2018-06-18 08:32:52.016219] DEBUG: service.fit: Initialize Fit class
[2018-06-18 08:34:19.191647] DEBUG: gui.statsPane: Setting full view for: resources

Which doesn't really help (a lot of things happen in between Fit class and the stat init... :/)

EDIT: Doesn't really help in narrowing it down, but it's more data! Seems the other logs that I have also have a bit of a delay between these two logs as well, so probably something is going on. :)

@MisfitMarius yes, i mean anti-virus monitor.

it could be a regression introduced by 2.9.0 (fix for #1828, although load time increase on my laptop is only 12ms). Please try downloading 2.8.0 and see if it's better (rename <user folder>\.pyfa temporarily if it fails to start). If it is, download 2.9.0 and see if it has the regression. If it does not, try the latest build and see if you still have the issue with clean database (before moving your old data back).

Alternatively - if you know what git bisect is, how to use it and you're able to run pyfa from source - could use it instead.

Regarding background processes - I do not know what the classification is. Will look it up later.

My turn. I had a slow startup version previously but ignored it until it went away with a later version. I just installed the latest version (pyfa-v2.9.6-win.exe) and it is slow again. I watched in the task manager and pyfa slowly counts up to about 120MB and then it displays. I see some errors in the log file but don't know what they mean. I used a timer on my phone and it takes almost 90s to startup.
pyfa-2019-08-02.log

Errors are irrelevant, however exact timings can tell a thing or two.

This part could be better:

[2019-08-02 04:38:17.098203] INFO: __main__: Running in a frozen state.
[2019-08-02 04:38:21.629139] INFO: eos.db: Initializing database
[2019-08-02 04:38:21.629139] INFO: eos.db: Gamedata connection: sqlite:///D:\Program Files (x86)\pyfa\eve.db?check_same_thread=False
[2019-08-02 04:38:21.629139] INFO: eos.db: Saveddata connection: sqlite:///C:\Users\Eric\.pyfa\saveddata.db?check_same_thread=False
[2019-08-02 04:38:27.280319] DEBUG: service.prefetch: Run database migration.

But this is just awful:

[2019-08-02 04:38:27.358320] DEBUG: service.prefetch: Completed database validation.
[2019-08-02 04:38:31.180452] INFO: gui.bitmap_loader: Using local image files.
[2019-08-02 04:38:45.569105] DEBUG: service.market: Initialize ShipBrowserWorkerThread.
[2019-08-02 04:38:45.626109] DEBUG: service.fit: Initialize Fit class
[2019-08-02 04:39:13.427166] DEBUG: gui.statsPane: Setting full view for: resources

@EricE

For the reference, startup times from my machine:

[2019-08-07 05:33:56.289815] INFO: __main__: Running in a thawed state.
[2019-08-07 05:33:56.321459] INFO: eos.db: Initializing database
[2019-08-07 05:33:56.321507] INFO: eos.db: Gamedata connection: sqlite:////home/dfx/Workspace/eve/pyfa/eve.db?check_same_thread=False
[2019-08-07 05:33:56.321535] INFO: eos.db: Saveddata connection: sqlite:////home/dfx/.pyfa/saveddata.db?check_same_thread=False
[2019-08-07 05:33:56.399150] DEBUG: service.prefetch: Run database migration.
...
[2019-08-07 05:33:56.466767] DEBUG: service.prefetch: Completed database validation.
[2019-08-07 05:33:56.581923] INFO: gui.bitmap_loader: Using local image files.
[2019-08-07 05:33:56.658212] DEBUG: service.market: Initialize ShipBrowserWorkerThread.
[2019-08-07 05:33:56.696889] DEBUG: service.fit: Initialize Fit class
[2019-08-07 05:33:57.099069] DEBUG: gui.statsPane: Setting full view for: resources

It takes under 1 second to get to the last line while in your log it takes pyfa almost a minute.

@EricE which antivirus/scanning/monitoring software are you using?

Hmm, just started up 2.9.6 and it started very quickly... I tried twice and it was great both times. I'm using Kaspersky Internet Security 19. No changes to my system that I can think of between the two reports.

pyfa-2019-08-09.log

My guess is that as AV monitor is collecting data about executable into global databases, it is monitoring pyfa in "strict" mode which slows down certain operations.

I will try to add more logging between lines I posted above to see what exactly is causing it.

Add some extra logging to help with #1605 debugging

I've added some extra logging in 9803da1. No idea if it will be enough, but atm I do not feel like I am ready to log every function call.

@MisfitMarius https://ci.appveyor.com/project/blitzmann/pyfa/builds/26758783/artifacts
Download one of those builds, launch and see if it still launches so long. If it is, provide log file.

(I have not tested if these builds work yet btw, this is still work in progress so they might be broken)

that's funny because I did not change anything for startup lag. Thanks for logs, i will check them later

Navigation Menu

Repository navigation

Start up lag #1605

If you're experiencing this issue

13 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Start up lag #1605

Description

If you're experiencing this issue

Activity

Calavoow commented on May 31, 2018

blitzmann commented on May 31, 2018

Calavoow commented on Jun 1, 2018

blitzmann commented on Jun 1, 2018

mosni commented on Jun 1, 2018

blitzmann commented on Jun 1, 2018

blitzmann commented on Jun 2, 2018

fsufitch commented on Jun 4, 2018

blitzmann commented on Jun 4, 2018

fsufitch commented on Jun 4, 2018

blitzmann commented on Jun 4, 2018

fsufitch commented on Jun 5, 2018

Leandrid commented on Jun 18, 2018

blitzmann commented on Jun 18, 2018

13 remaining items

DarkFenX commented on May 22, 2019

EricE commented on Aug 2, 2019

DarkFenX commented on Aug 7, 2019

DarkFenX commented on Aug 7, 2019

EricE commented on Aug 9, 2019

DarkFenX commented on Aug 9, 2019

DarkFenX commented on Aug 12, 2019

DarkFenX commented on Aug 17, 2019

DarkFenX commented on Aug 23, 2019

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions