Fine Tuning FAQs
How long will it take for my job to start?
It depends. Factors that affect waiting time include the number of number of pending jobs from other customers, the number of jobs currently running, and available hardware. If there are no other pending jobs and there is available hardware, your job should start within a minute of submission. Typically jobs will start within an hour of submission. However, there is no guarantee on waiting time.
How long will my job take to run?
It depends. Factors that impact your job run time are model size, training data size, and network conditions when downloading/uploading model/training files. You can estimate how long your job will take to complete training by multiplying the number of epochs by the time to complete the first epoch.
Why am I getting an error when uploading a training file?
There are two common issues you may encounter,
Your API key may be incorrect. If you get a 403 status code, this indicates your API Key is incorrect.
Your balance may be less than the job minimum. We verify that you have sufficient balance on your account that is equal to the minimum job charge ($5). If you do not have sufficient balance, you can increase your account limit by adding a credit card to your account, adjusting your spending limit if you ready have a credit card, or paying your outstanding account balance. If you have sufficient balance on your account, contact support for assistance.
Why was my job cancelled?
There are two reasons that a job may be automatically cancelled.
You do not have sufficient balance on your account to cover the cost of the job.
You have entered an incorrect WandB API key
You can determine why your job was cancelled by:
(1) checking the events list for your job via the bytecompute-CLI tool
$ bytecompute list-events <job-fine-tune-id>
(2) Via the web interface https://bytecompute.ai/docs?id=openai_api > Jobs > Cancelled Job > Events List
What should I do if my job is cancelled due to billing limits?
You can an add a credit card to your account to increase your spending limit. If you already have a credit card on your account, you can make a payment or adjust your spending limit. Contact support if you need assistance with your account balance.
Can I download the weights of my model?
If you would like to download the weights of your model so that you can use your fine-tuned model outside of our platform, you are able to do this through the following:
To download the weights of a fine-tuned model, run:
bytecompute fine-tuning download <FT-ID>
This command will download ZSTD compressed weights of the model. To extract the weights, run tar -xf filename.
Other arguments:
--output,-o (filename, optional) -- Specify the output filename. Default: <MODEL-NAME>.tar.zst
--step,-s (integer, optional) -- Download a specific checkpoint's weights. Defaults to download the latest weights. Default: -1