SEFurkan GözükaraSECourses · Generative AI Join onPatreon

Multi-GPU SDXL Training Speed Test With Kohya on 8x A100 GPUs

Multi-GPU training speed difference research: SDXL training with Kohya SS GUI on an 8x A100 SXM4 machine on Massed Compute, SXM vs PCIe linked GPUs.

Update 1 while still researching: a single GPU trains with batch size 7 but more than 1 GPU fails, and the issue was reported to Kohya. Includes the single GPU training speed screenshot.

Updated By Dr. Furkan Gözükara
Massed ComputeComparison
Multi-GPU SDXL Training Speed Test With Kohya on 8x A100 GPUs
Older post. this is an older post kept for reference. Newer apps and guides are in Research, Comparisons and Experiments.

Rented this machine on Massed Compute to test Stable Diffusion XL training speed difference with Kohya SS GUI on a 8x A100 (SXM4) machine and see the difference of multi-GPU training SXM vs PCIe linked GPUs

Hopefully will update this post as I get more info.

Multi-GPU SDXL Training Speed Test With Kohya on 8x A100 GPUs
 

Update 1

Single GPU works with Batch size 7 but more than 1 GPU fails

Reported Kohya : https://github.com/kohya-ss/sd-scripts/issues/1434#issuecomment-2244049699

Single GPU training speed as below with batch size 7

Update 1 – Multi-GPU SDXL Training Speed Test With Kohya on 8x A100 GPUs
 

↑ ↓ to move · Enter to open · Esc to close · Ctrl K or / to search from any page