Automatic Polygon(Mask) Drawing

Implemented a Modified Segment Anything Model to facilitate real-time user interaction (50ms to 200ms latency) for segmenting large TIFF images, generating precise polygon masks, and improving the efficiency and accuracy of polygon drawing tools.

Segment Anything Model (SAM) is notable for its ability to understand a general notion of objects and generate masks for any object in it.

Traditional approaches required either interactive segmentation, which is dependent on human input for refining masks, or automatic segmentation, which is limited to predefined object categories and extensive manual annotation. SAM overcomes these limitations by being a single model capable of performing both interactive and automatic segmentation. Its promptable interface allows for a wide range of segmentation tasks through engineered prompts, like clicks, boxes, text, etc.

The model has been optimized to run in real-time on a CPU in a web browser, which is crucial for interactive use. But it was not the case for the larger image. While running the SAM for a larger image size it took an hour which was the challenge.

By modifying the architecture(confidential) of the SAM I was able to perform the real-time segmentation in the large geo-image with a size of around 300MB.