No. The image you see in Finder’s Get Info pane is of the View pane at save/close. However, I don’t believe there is a method allowing quicklook to see deeper into the file.
FWIW, ‘preview’ is a slippery term here. In a TBX with a few notes and no agents, action code etc., and using no Displayed Attributes, then previewing a note might be as simple as its title and body text. In reality, many docs are much more complex so you can’t ‘just’ look at a note and see what you see in the app. I assume this is why Finder/quicklook shows the information it does.
If you just need to see the data, simply open the TBX in BBEdit or the like - the TBX is simply a large well-formed XML document so can be read as XML (though the rich text aspects of $Text are stored in a further encoded in Base64 format (see). Inline images in text are stored in the rich text data.