01
Find the earlier reader
Trace code that previews, validates, hashes, or parses the same file-like object before upload. Reading consumes stream data and changes the cursor. The buffer can contain the correct report while its current readable position is already at the end.
In the reproduction, a BytesIO buffer held a short report. Calling read consumed it. upload_from_file without a supplied size then selected a resumable upload path and rejected the stream’s nonzero position before initiating an upload.
02
Inspect position without consuming more data
For a seekable stream, tell reports the current position. Compare it with your application’s intended start position. Avoid adding another read to a diagnostic and accidentally reproducing the same problem.
Not every stream supports seeking. A socket, pipe, or incoming request body may be sequential. In that case, choose a bounded buffering or streaming design appropriate to the payload instead of assuming seek(0) will work everywhere.
from io import BytesIO
payload = b"complete report\n"
stream = BytesIO(payload)
stream.read() # A preview consumes the stream.
assert stream.tell() == len(payload)03
Reset the seekable stream before upload
Use rewind=True when the complete content should be read from its beginning. The following small-buffer helper supplies a size already known from the payload; it is not a reason to load a large or unbounded file into memory.
The create-only generation condition also prevents this example from replacing an existing live object. Select the real destination and write semantics deliberately. A cursor repair and an overwrite policy solve different problems.
def upload_small_report(blob, payload):
from io import BytesIO
stream = BytesIO(payload)
stream.read() # Simulate earlier validation or preview.
blob.upload_from_file(
stream,
rewind=True,
size=len(payload),
content_type="text/plain",
if_generation_match=0,
)04
Check exactly which bytes were sent
Our local fixture captured the SDK’s multipart body after the repair and compared its data part with the complete report. The stream failure made no upload request; the repaired call sent all expected bytes.
On a real authorized test destination, verify the expected object size and contents or an appropriate checksum. An object name appearing in a listing is insufficient evidence that a previously consumed stream uploaded the intended report.
05
Keep stream ownership clear
If several components share one file object, define which component owns its cursor and whether each read must restore position. Reopening an ordinary file in binary mode for upload can be clearer than passing around a partially consumed handle.
Capture the library version and whether the call used a known size or resumable path. Do not generalize the observed ValueError to every upload mode: different paths can consume the stream differently. The repair should match the intended byte range and the stream’s capabilities.
Know the limits
Reproduced with google-cloud-storage 3.17.1 and BytesIO. A consumed stream on the SDK’s resumable path raised ValueError before upload; rewind=True with a known small size produced the full multipart payload in a local HTTP fixture. No live upload or resumable network session was tested.
Sources & further reading
A technical publication by CoreTrailKit.
Browse all articles