Skip to content

attempt to yield across C-call boundary in function 'require' with Openresty #78

Closed

Description

@Blasterdick

With uncommented line:

local http_request = require("http.request")

I'm seeing this error, while line:

local http_headers = require("http.headers")

doesn't lead to it.

I'm using docker openresty:alpine inage, lua-http installed with luarocks5.1.

lua_package_path "/usr/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?.lua;;";

Path to a /usr/local/share/lua/5.1/http/ provided with a symlink.

bash-4.3# ls -l /usr/local/share/lua/5.1/
total 56
-rw-r--r--    1 root     root          9559 Jul  5 11:07 basexx.lua
drwxr-xr-x    2 root     root          4096 Jul  5 11:06 compat53
drwxr-xr-x    3 root     root          4096 Jul  5 11:07 cqueues
-rw-r--r--    1 root     root          4105 Jul  5 11:07 cqueues.lua
-rw-r--r--    1 root     root          2097 Jul  5 11:07 fifo.lua
lrwxrwxrwx    1 root     root            41 Jul  5 11:07 http -> /usr/local/share/lua/5.1/http_0_2_0-http/
drwxr-xr-x    3 root     root          4096 Jul  5 11:07 http_0_2_0-http
drwxr-xr-x    2 root     root          4096 Jul  5 11:07 lpeg_patterns
drwxr-xr-x    4 root     root          4096 Jul  5 11:07 openssl
-rw-r--r--    1 root     root            25 Jul  5 11:07 openssl.lua
-rw-r--r--    1 root     root          6286 Jul  5 11:07 re.lua

Activity

  1. daurnimator commented on Jul 6, 2017

    @daurnimator
    Owner

    Could you give me a traceback? I'm curious where this is coming from.

  2. Blasterdick commented on Jul 6, 2017

    @Blasterdick
    Author
    2017/07/06 12:37:19 [error] 14#14: *1 lua entry thread aborted: runtime error: attempt to yield across C-call boundary
    stack traceback:
    coroutine 0:
            [C]: in function 'require'
            /var/www/config/nginx/lua/google_pubsub.lua:8: in function </var/www/config/nginx/lua/google_pubsub.lua:1>,
    

    I'm using lua-http to sent push (POST) requests to Google Pub/Sub with function like this:

    local http_request = require("http.request")
    local http_headers = require("http.headers")
    
    local req2 = function(uri, body, auth)
        local headers = http_headers.new()
        headers:append(":authorization", auth)
        headers:append(":content-length", #body)
    
        local request = http_request.new_from_uri(uri)
        request.version = 2
        request.tls = true
        request.port = 443
        request.headers = headers
        request:set_body(body)
    
        local resp_headers, stream = request:go()
        if resp_headers:get ":status" ~= "200" then
            ngx.log(ngx.ERR, stream:get_body_as_string())
        end
    end
    

    Also, here's a Dockerfile snippet with deps installing process:

    FROM openresty/openresty:alpine
    RUN \
    	apk update && \
    	apk add --no-cache unzip gcc make linux-headers bsd-compat-headers musl-dev m4
    RUN \
    	apk add --no-cache openssl openssl-dev lua5.1 lua5.1-dev luajit luarocks5.1 lua5.1-cjson lua5.1-socket lua5.1-sec lua5.1-crypto lua5.1-lpeg
    RUN \
        luarocks-5.1 install http && \
        ln -s /usr/local/share/lua/5.1/http_0_2_0-http/ /usr/local/share/lua/5.1/http
    

    Here's nginx.conf snippet of lua_package_path:

    lua_package_path "/usr/share/lua/5.1/?.lua;/usr/local/share/lua/5.1/?.lua;;";
    lua_package_cpath "/usr/lib/lua/5.1/?.so;/usr/local/lib/lua/5.1/?.so;;";
    
  3. daurnimator commented on Jul 7, 2017

    @daurnimator
    Owner

    I'm using lua-http to sent push (POST) requests to Google Pub/Sub with function like this:

    Note that cqueues (and hence lua-http) doesn't currently play nice with openresty: :go() and other lua-http blocking operations will block the nginx main loop. This is something I've tried to fix before ( see openresty/lua-nginx-module#450)

    I'm using lua-http to sent push (POST) requests to Google Pub/Sub with function like this:
    ...
    Also, here's a Dockerfile snippet with deps installing process:

    What do I do with these? Could you give me complete instructions to replicate?

    ln -s /usr/local/share/lua/5.1/http_0_2_0-http/ /usr/local/share/lua/5.1/http

    It's odd that you need this....

  4. Blasterdick commented on Jul 7, 2017

    @Blasterdick
    Author

    What do I do with these? Could you give me complete instructions to replicate?

    I thought, maybe some packages have wrong versions or are missing.

    It's odd that you need this....

    Yup, but that's the way luarocks install this module in the alpine linux.

    will block the nginx main loop

    So, it's a bad idea for me to use Openresty for this purpose, or bare-metal Nginx with some special modules should do the trick? (sorry for a newbie question) Maybe, I should give some other web server a spin in that case?

  5. daurnimator commented on Jul 8, 2017

    @daurnimator
    Owner

    What do I do with these? Could you give me complete instructions to replicate?

    I thought, maybe some packages have wrong versions or are missing.

    I mean: can you give me a sequence of commands to type into my shell to make that docker snippet + lua snippet throw the error? i.e. give me the full docker file; give me a full nginx.conf; etc.
    It takes me far longer if I have to read through how the docker image you use is created+how it's meant to be configured, and then code up 100 lines of supporting config+code just to make your snippet run.

    Yup, but that's the way luarocks install this module in the alpine linux.

    Has this been brought up with the alpine luarocks maintainer? @jirutka ping.

    So, it's a bad idea for me to use Openresty for this purpose,

    It won't be performant if you use the default lua-http blocking API.
    One solution is to not use any blocking APIs. i.e. you'd need to call things with a timeout of 0. This mode of use isn't fully fleshed out in the current lua-http release (e.g. there's no request-in-progress object (see https://github.com/daurnimator/lua-http/projects/2#card-552050)); but you should be able to get something basic working.

    bare-metal Nginx with some special modules should do the trick

    I'm not sure if there is a nginx http2 client module that supports this type of usage.

    Maybe, I should give some other web server a spin in that case?

    Maybe. I don't fully know what you're trying to do.
    You could use lua-http to create a minimal http server (see https://github.com/daurnimator/lua-http/blob/master/examples/server_hello.lua) and then use it as an upstream from your nginx instance.

  6. Blasterdick commented on Jul 12, 2017

    @Blasterdick
    Author

    Thank you for a suggestion, now I'm trying to use http-lua as a server, and I'm getting this error:

    Pseudo-header fields are only valid in the context in which they are defined

    with next code:

    -- Make an HTTP/2.0 request
    local req2 = function(uri, body, auth)
    
        local headers = http_headers.new()
        headers:upsert(":method", "POST")
        headers:upsert(":scheme", "https")
        headers:upsert(":path", uri)
        headers:upsert(":authorization", auth)
        headers:upsert(":content-length", #body)
    
        local request = http_request.new_from_uri(uri)
        request.version = 2
        request.tls = true
        request.port = 443
        request.headers = headers
        request:set_body(body)
    
        local resp_headers, stream = request:go(30)
        if resp_headers:get ":status" ~= "200" then
            print(stream:get_body_as_string())
        end
    end
    
  7. daurnimator commented on Jul 12, 2017

    @daurnimator
    Owner

    Pseudo-header fields are only valid in the context in which they are defined

    headers:upsert(":authorization", auth)
    headers:upsert(":content-length", #body)

    ":authorization" and ":content-length" are not a valid pseudo headers (you probably didn't mean to write the colon).

    The only headers that can start with : are :path, :method, :authority, :scheme and :status. See http://httpwg.org/specs/rfc7540.html#PseudoHeaderFields for more info.

    request.headers = headers

    Avoid replacing the headers field: instead modify the existing one.
    If you do so, you can avoid upserting scheme and path.

  8. Blasterdick commented on Jul 13, 2017

    @Blasterdick
    Author

    It errors with attempt to index a nil value (field 'headers') if I try to add headers to an existing one.

    Won't you, please, review my code? I'm a complete newbie in Lua and not a programmer at all, so only with a little help from my friends I can get something working.

    Thanks in advance. comrade.

  9. Blasterdick commented on Jul 13, 2017

    @Blasterdick
    Author

    After some editing, I came up with this version.

    Now it hangs till timeout and errors with:

    failed: pubsub_push.lua:17: attempt to index a nil value (local 'resp_headers')

    While netstat shows an open connection to Google PubSub IP from appropriate PID.

  10. rokf commented on Jul 19, 2017

    @rokf

    The error happens because of the timeout. On timeout the go method returns nil, the error message and error number so your resp_headers variable gets the value nil. I'd say that there's something wrong with the headers. The possible errors are described here (section 3.1.3) if you're interested. Have you tried doing the same request with some other tool like curl?

  11. Blasterdick commented on Jul 19, 2017

    @Blasterdick
    Author

    Thanks for a suggestion, comrade. Yup, I've tried with Postman with the same headers and raw POST data.

    Duplicated question here.

  12. rokf commented on Jul 19, 2017

    @rokf

    Does your request body have the correct JSON format like required?(https://cloud.google.com/pubsub/docs/reference/rest/v1/projects.topics/publish)

  13. Blasterdick commented on Jul 19, 2017

    @Blasterdick
    Author

    Affirmative. With Postman everything works as expected.

  14. daurnimator commented on Jul 24, 2017

    @daurnimator
    Owner

    I was able to make your code work.

    Notes:

    • Google responds with a RST_STREAM if your token contains a new line. consider using "*l" to read your token so you know you don't get a newline
    • Consider caching your token rather than reading it every time
    • Use assert with most lua-http calls (or handle your errors)
      • Around :get_body_as_string()
      • Around :go()
      • When using the lua file api, e.g. io.open
  15. daurnimator commented on Jul 26, 2017

    @daurnimator
    Owner

    @Blasterdick okay to close?

  16. Blasterdick commented on Jul 27, 2017

    @Blasterdick
    Author

    @daurnimator For now I'm using the code, adapted for my needs from this article.

    Thank you anyway, I'll try to modify my Lua code, and would give it a spin as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions