HTTP GET returns no data

SDK Get Help
1

Hi! I’m working on a web browser for the playdate, but I’m having issues with HTTP requests. In particular, I can retrieve data from sites like example.com and info.cern.ch, but not google or text.npr.org

I’ve uploaded a MRE to show this (code at bottom, for some reason I can’t link to github)

Running the code in the simulator produces the logs (it seems there are bytes available from the manual poll, but for some reason the callbacks are not called and `headersRead` returns a 0 status code):

[3096] 	HTTP GET getBytesAvailable: 0
[3097] HTTP Waiting 100 ms for data: OK
[3236] 	HTTP GET getBytesAvailable: 0
[3238] HTTP Waiting 100 ms for data: OK
[3256] HTTP headersRead called
[3257] 	HTTP GET getResponseStatus: 0
[3258] 	HTTP GET getBytesAvailable: 0
[3377] 	HTTP GET getBytesAvailable: 6373
[3378] HTTP Waiting 100 ms for data: OK
[3517] 	HTTP GET getBytesAvailable: 6373
[3519] HTTP Waiting 100 ms for data: OK

But changing google to example.com works.

import "CoreLibs/utilities/where"
import "CoreLibs/object"

local net <const> = playdate.network

playdate.display.setRefreshRate(50)

-- set url to google.com will fail
local url = "google.com"
local http_done = false
local http_result = ""
local http_waiting = false
local http_data_received = false
local http_conn = nil
local start_time <const> = playdate.getCurrentTimeMilliseconds()

function timeLog(text)
    now = playdate.getCurrentTimeMilliseconds()
    elapsed = now - start_time
    print(string.format("[%i] %s", elapsed, text))
end

function headersRead()
    timeLog("HTTP headersRead called")

    local response = http_conn:getResponseStatus()
    timeLog(string.format("\tHTTP GET getResponseStatus: %i", response))
end

function connectionClosed()
    timeLog("HTTP connectionClosed called")
end

function requestComplete()
    local bytes = http_conn:getBytesAvailable()
    timeLog(string.format("\tHTTP GET getBytesAvailable: %i", bytes))
end

function requestCallback()
    timeLog("HTTP requestCallback called, "..http_conn:getBytesAvailable().." bytes are available for reading")
    local bytes = http_conn:getBytesAvailable()
    local data = http_conn:read(bytes)
    print(data)
    http_data_received = true
end

function playdate.update()

    if not http_done then
        if not http_waiting then

            http_conn = net.http.new(url, nil, true, "HTTP Demo")
            assert(http_conn, "The user needs to allows this")

            http_conn:setHeadersReadCallback(headersRead)
            http_conn:setConnectionClosedCallback(connectionClosed)
            http_conn:setRequestCompleteCallback(requestComplete)
            http_conn:setRequestCallback(requestCallback)

            http_conn:setConnectTimeout(2)
            http_conn:setKeepAlive(true)

            local get_request, get_error = http_conn:get("/")
            assert(get_request, get_error)

            http_waiting = true

        else
            if http_data_received then
                http_conn:close()
                http_done = true
            else
                local bytes = http_conn:getBytesAvailable()
                timeLog(string.format("\tHTTP GET getBytesAvailable: %i", bytes))

                local wait_time = 100
                timeLog(string.format("HTTP Waiting %i ms for data: %s", wait_time, http_conn:getError() or "OK"))
                playdate.wait(wait_time)
            end
        end
    end

end
3

if you check bytes available in the request complete callback. and read them out, do you get your results? I have a sneaking suspicion that the data available callback isn’t called if the connections readbuffer never gets filled before the request finishes.

4

The request complete callback isn’t seeing data either, but manually polling shows bytes available after the header read callback is called (which returned 0 status code). I realized I need to wait for data to arrive and updated the code accordingly.

5

Actually I think you might be right about the buffer getting filled after the request finishes: calling getBytesAvailable sees data after the request complete callback returns.