Patreon
Home / Apps / Captioning, Dataset and Image Processing Tools / Image Validator Script for Training Datasets: Move Corrupted

Image Validator Script for Training Datasets: Move Corrupted Images

Image validator script that scans a training dataset folder and its subfolders, logs invalid images and moves corrupted ones that would break training.

dataset toolswindowslinuxrunpod
Image Validator Script for Training Datasets: Move Corrupted Images

Highlights

Made after a supporter had corrupted images among 100k+ training images. Set folder_path, broken_folder_path and move_broken, then run the .bat file on Windows or the Python script on Linux or RunPod.

What it does

Corrupted images can break training. This script tests every image in a folder and its subfolders, writes the invalid ones to an invalid_images_log.txt file so you can find and fix broken images, and moves corrupted images into a separate folder. Moving is enabled by default and can be turned off.

It was written when a supporter had corrupted images among more than 100,000 training images.

What you get

Requirements

Runs on: Windows, Linux, RunPod. Installers are 1-click and use an isolated Python environment, so they do not break other apps.
Where to get it: the installers, presets and full instructions are in this Patreon post by Furkan Gözükara (SECourses). Published Jul 05, 2023.

Related

Ultimate Image Processing App: 1-Click Batch Cropping, SAM 3.1 Masking

Ultimate Image Processing App: 1-Click Batch Cropping, SAM 3.1 Masking

Ultimate Image Processing Premium turns raw photo folders into training-ready datasets with nine tools, from SAM 3.1 cropping to duplicate removal.

All JoyCaption Apps: 1-Click Installers With Batch Captioning

All JoyCaption Apps: 1-Click Installers With Batch Captioning

JoyCaption in four improved Gradio apps: Pre-Alpha, Alpha One, Alpha Two and the newest, fastest Joy Caption Beta One, all with batch and multi-GPU captioning.

Ultimate Image Captioner Pro: Qwen3 VL & JoyCaption JSON Captioning

Ultimate Image Captioner Pro: Qwen3 VL & JoyCaption JSON Captioning

Ultimate Image Captioner Pro writes JSON and text captions with Qwen3 VL 8B Instruct and Joy Caption models for Ideogram 4 JSON prompting and training.

More in Captioning, Dataset and Image Processing Tools.