← All Updates
On-Prem Agent

Dataset Upload Pre-Flight Validation

August 21, 2026

If files or folders are moved or deleted after a dataset is registered but before it uploads, the upload can no longer succeed — but until now that situation looked the same as a temporary network problem, and the agent would keep retrying.

The agent now runs a pre-flight check on a dataset's files and folders before uploading. If anything is missing, the dataset is marked invalid with a clear error message explaining what couldn't be found, and it stops consuming retry attempts. Genuinely transient filesystem and network errors still retry as before, so the change only affects datasets that can never complete.

Details

  • Missing files are detected before and during upload
  • Invalidated datasets carry an explicit error message rather than failing silently
  • Permanently missing files no longer trigger repeated retries
  • Transient filesystem and upload errors remain retryable
  • Subsequent dataset events continue processing without duplicate errors