Notes:
Parameter Settings:
Set Mixed Precision Type to BF16.
Don't use 8 bit Optimizers. You can use any non 8 bit Optimizer.
From Paramaters -> Advanced, set CrossAttention to SDPA.
No xFormers support for now.
Only BF16 and FP32 are supported. FP16 support for training isn't complete yet.
No...