Skip to content

server tests : more pythonic process management; fix bare except: - #6146

Merged
phymbert merged 9 commits into
masterfrom
ceb/pythonic-server-tests
Mar 20, 2024
Merged

phymbert merged 9 commits into
masterfrom
ceb/pythonic-server-tests

Conversation

@cebtenzzre

@cebtenzzre cebtenzzre commented Mar 18, 2024 •

Copy link
Copy Markdown
Collaborator

ref #6098 (comment)

This code has been consistently confusing to me. Here I suggest a few simplifications:

  • Don't add an extra \n to a random selection of print()s. A habit from C perhaps? Let me know if any (or all) of these are intentional - it just looked strange to me to have a blank line after so many prints.
  • I changed the process management code in the server tests to use built-in subprocess functions instead of os.kill or psutil.
  • Before we were sending SIGINT, waiting 50ms, then sending SIGKILL in a loop, and after several tries also sending SIGKILL with both native Python and psutil at the same time. This seems really odd to me, so I changed it to just send SIGKILL if SIGINT didn't kill the process after 500ms. Maybe Popen.terminate() was intended as the second stage of killing, but this will have the same effect as SIGKILL for the server process, even on Unix, as it is not explicitly handled via sigaction.
  • I also fixed the neighboring except: clause - it's wrong to ignore exceptions like KeyboardInterrupt and SystemExit, so I changed it to except Exception:. And traceback.print_exc seemed more appropriate here.

@cebtenzzre
cebtenzzre requested a review from phymbert March 18, 2024 18:39

@phymbert phymbert left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably better code but the build failed... maybe you must test it before

@cebtenzzre

cebtenzzre commented Mar 19, 2024 •

Copy link
Copy Markdown
Collaborator Author

I was able to run the tests locally after making the connect routine match what the server does - it uses the first address family returned by getaddrinfo, whether it is ipv4 or ipv6. Otherwise, the connection failed because the server was listening on ::1 and the client was trying to connect to 127.0.0.1.

see also #5372 (comment)

@cebtenzzre
cebtenzzre requested a review from phymbert March 19, 2024 15:40
@phymbert

Copy link
Copy Markdown
Collaborator

Otherwise, the connection failed because the server was listening on ::1 and the client was trying to connect to 127.0.0.1.

There is a hack on windows server test to force the server to listen 0.0.0.0. But it's probably not the same issue you faced with ipv6

@phymbert phymbert left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this, your contribution on server tests is highly appreciated 👍
If you don't mind I will request your review on changes related to python server tests.

I have included fixes for:

All tests are now working and the python code is clearer, great!

@phymbert
phymbert merged commit bd60d82 into master Mar 20, 2024
@phymbert
phymbert deleted the ceb/pythonic-server-tests branch March 20, 2024 05:34
hodlen pushed a commit to hodlen/llama.cpp that referenced this pull request Apr 3, 2024
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request Apr 26, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
phuongncn pushed a commit to phuongncn/llama.cpp-gx10-dgx-sparks-deepseekv4 that referenced this pull request Apr 28, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request May 6, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request May 15, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request Jun 2, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
fukuro-kun pushed a commit to fukuro-kun/fukuro-llama-cpp-turboquant that referenced this pull request Jul 5, 2026
…gml-org#6146)

* server tests : remove seemingly redundant newlines in print()

* server tests : use built-in subprocess features, not os.kill and psutil

* server tests : do not catch e.g. SystemExit; use print_exc

* server tests: handle TimeoutExpired exception

* server tests: fix connect on dual-stack systems

* server: tests: add new tokens regex on windows generated following new repeat penalties default changed in (ggml-org#6127)

* server: tests: remove the hack on windows since now we get the good socket family

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

* server: tests: add new tokens regex following new repeat penalties default changed in (ggml-org#6127)

---------

Co-authored-by: Pierrick HYMBERT <pierrick.hymbert@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants