SegRef3D

SegAnything: SegRef3D Job Workflow

SegAnything uses a Google Colab GPU as a computation backend. All prompt configuration is performed in SegRef3D; there is no separate Gradio interface.

1. Configure Objects In SegRef3D

1. Load the image sequence in SegRef3D Local GPU, Local CPU, or SegRef3D Lite.
2. Select an object ID under Target Object.
3. Move to a frame where the object is clear and choose Set Box Prompt.
4. Draw the box around the object. This frame becomes its Prompt Frame.
5. Move to the first desired frame and choose Set Tracking Start.
6. Move to the last desired frame and choose Set Tracking End.
7. Choose Add Object Prompt.
8. Repeat for other objects.

Open Extensions > Batch Tracking > Batch Jobs to review names, Prompt Frames, Tracking Ranges, and box coordinates. Prompt Frame must be inside its Tracking Range.

2. Export The Job

Under AI Segmentation > SegAnything, choose Create Input ZIP and save seganything_request.zip. It contains the working JPG sequence and manifest.json.

3. Run SegAnything

1. Choose SegAnything in SegRef3D, or open SegAnything.
2. In Colab, select Runtime > Change runtime type > T4 GPU > Save.
3. Select Runtime > Run all.
4. Upload seganything_request.zip when the upload control appears.
5. Leave the notebook open while it processes every object forward and backward.
6. At Segmentation complete, the separate final cell automatically starts downloading seganything_result.zip through the browser.

The progress display shows the current step, object, frame, and overall progress.

4. Import The Result

1. Return to SegRef3D.
2. Under AI Segmentation > SegAnything, choose Import Result ZIP.
3. Select seganything_result.zip.
4. Confirm replacement if the current project already contains label masks.

SegRef3D validates the image sequence and masks before applying them. If no images are currently loaded, the JPG sequence in the result ZIP is restored automatically. Imported masks are immediately written to a new [autosave] label PNG folder.

SegRef3D Lite controls

The browser app uses the same manifest and ZIP formats. Open SegAnything > Edit Setup to enter AI Tracking Setup, set each object’s prompt and range, choose Create Input ZIP, then restore the Colab output with Import AI Result. Keep AI Tracking Setup open while navigating with the mouse wheel or F/R, then use Use current to capture Tracking start and end. Add Box Prompt Here displays crosshair guides for accurate Box Prompt placement. No local SAM2 package or NVIDIA GPU is required for job preparation.

5. Refine And Reconstruct

Use the normal SegRef3D tools to add, erase, transfer, relabel, interpolate, measure, and export NIfTI, TIFF, STL, overlay PNG, or volume CSV files.

The mask PNGs are single-channel label images: 0 is background and 1 through 20 are object IDs.

Troubleshooting

• Invalid ZIP or missing manifest: export the job again from SegRef3D.
• Prompt Frame outside Tracking Range: edit the object in Batch Jobs.
• CUDA/model error: confirm that the Colab runtime is using a T4 or another GPU, then choose Runtime > Disconnect and delete runtime and run all cells again.
• Interrupted processing: rerun all cells, upload the same input ZIP in the first executable cell, then leave the notebook open while the remaining cells continue.
• Download did not start: run only the final download cell again. It uses Colab’s native files.download() instead of a localhost notebook link.

This site is open source. Improve this page.