FAQ

Caveats and FAQ¶

• I'm seeing a prompt asking me to restart as part of the install

On certain newer distributions of Linux, you may be prompted as part of cometctl init or cometctl install with an ncurses prompt asking whether you'd like to restart any services. If you do not know what to do here, you may hit enter and bypass this.

• The curl command fails to install

Certain distributions will prompt for a confirmation to install the packages pulled in by the script pulled in by the first command. Save the script first and run it as a separate command and hit Y when prompted to install the packages.

• How can I verify the health of my application deployment?

One straightforward method is by accessing the following URL: http://your-deployment-host/api/isAlive/ping. You should receive a JSON response like this:

{"msg":"Healthy Server","code":200,"data":null,"sdk_error_code":0}

• How can I obtain or assign the Admin role within the Comet application?

By default, the first user who signs up for the Comet application is granted the Admin role. Once this initial Admin account is established, it has the privilege to assign the Admin role to other users via the Admin UI.

• What should I do if we lose access to our Admin accounts?*

If you find yourself in this situation, you can utilize the ADMIN_USER_LIST:['username'] environment variable. This can be set in /etc/default/comet-ml or under the additionalConfigurations: section of your values.yaml helm chart. After setting the variable, make sure to restart the backend pods to apply the changes.

• Why do I get a 500 or 403 error when uploading large files using log_model or log_asset_data?

Comet's backend uses multi-part upload for files larger than approximately 8MB. If your S3 or GCS storage bucket is missing the required permissions for multi-part uploads, you may encounter errors such as:

COMET ERROR: Uploading file failed (500) with max retries: 4
…
{“msg”:“failed to save asset.”,“code”:500,…}

Or:

AmazonS3Exception: Access Denied
Status Code: 403

To resolve this, ensure your storage bucket allows multi-part upload operations:

For AWS S3, verify that the following IAM permissions are granted:

[
  "s3:PutObject",
  "s3:PutObjectAcl",
  "s3:InitiateMultipartUpload",
  "s3:UploadPart",
  "s3:CompleteMultipartUpload",
  "s3:AbortMultipartUpload",
  "s3:ListMultipartUploadParts"
]

For Google Cloud Storage (GCS) using the S3-compatible XML API, see:
👉 https://cloud.google.com/storage/docs/access-control/iam-permissions#multipart-uploads
Make sure your bucket policy or IAM role does not explicitly block InitiateMultipartUpload or UploadPart.

Alternative workaround:
If adjusting permissions is not possible, consider using:

experiment.log_remote_model()

or

experiment.log_remote_asset()

to store large models or files directly in your own bucket without routing through Comet.

For more, see:
https://www.comet.com/docs/v2/guides/experiment-management/log-data/remote-data/