Sorry for the late reply--it's been a little busy over here! 
An odd thing about setBackgroundDrawingCallback() is that the sprite it creates ignores the global draw offset. (The source is in the SDK in CoreLibs/sprites.lua if you haven't seen it, might be useful to check it out.) We mention this in the docs but it's easy to miss:
The background sprite ignores the
drawOffset, and will not be automatically redrawn when the draw offset changes; useplaydate.graphics.sprite.redrawBackground()if necessary in this case.
Shifting by the offset in the callback should work, as long as you're invalidating the background sprite when you change the offset so that it knows to redraw. But an easier way to do this is just create a sprite for the background, give it a low Z index, then set the tilemap on it:
bg = gfx.sprite.new()
bg:setZIndex(-32767)
bg:setTilemap(tilemap)
bg:setCenter(0,0)
bg:moveTo(0,0)
bg:add()