Using custom modal with line_zone

[ravinpatel]

Hi,

I am trying to use my own custom modal with line_zone.trigger, it paints the line on the video however it doesn't pick up any objects moving past the line.

Here is my code:

the modal is first built like this:

rf = Roboflow(api_key="API_KEY")
project = rf.workspace().project("project_name")
version = project.version(2)
model = version.model

Here is me tracking the detections which work fine however line zone is never triggered and nothing is counted:

INPUT_VIDEO = "video.mp4"

job_id, signed_url, expire_time = model.predict_video(
   INPUT_VIDEO,
   fps=5,
    
   prediction_type="batch-video",
)

START = sv.Point(800, 0)
END = sv.Point(800, 1686)

line_zone = sv.LineZone(start=START, end=END)


line_zone_annotator = sv.LineZoneAnnotator(
    thickness=4,
    text_thickness=4,
    text_scale=2
)

results = model.poll_until_video_results(job_id)
box_mask_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()
tracker = sv.ByteTrack()

cap = cv2.VideoCapture(INPUT_VIDEO)
frame_rate = cap.get(cv2.CAP_PROP_FPS)
cap.release()

def annotate_frame(frame: np.ndarray, frame_number: int) -> np.ndarray:
    try:
        time_offset = frame_number / frame_rate
        closest_time_offset = min(results['time_offset'], key=lambda t: abs(t - time_offset))
        index = results['time_offset'].index(closest_time_offset)
        detection_data = results['project_name'][index]

        roboflow_format = {
            "predictions": detection_data['predictions'],
            "image": {"width": frame.shape[1], "height": frame.shape[0]}
        }
        
        detections = sv.Detections.from_inference(roboflow_format)
        detections = tracker.update_with_detections(detections)

    except (IndexError, KeyError, ValueError) as e:
        detections = sv.Detections(xyxy=np.empty((0, 4)),
                                   confidence=np.empty(0.7),
                                   class_id=np.empty(0, dtype=int))

    annotated_frame = box_mask_annotator.annotate(frame.copy(), detections=detections)
    annotated_frame = label_annotator.annotate(annotated_frame, detections=detections)

    crossed_in, crossed_out = line_zone.trigger(detections)
    if np.any(crossed_in) or np.any(crossed_out):
        print("someone crossed")
    return line_zone_annotator.annotate(annotated_frame, line_counter=line_zone)

sv.process_video(
    source_path="/video.mp4",
    target_path="/video-output.mp4",
    callback=annotate_frame
)

supervision: 0.22.0

I'm quite new to CV and Roboflow but I tried to follow blogs/videos but no results.

Thank you in advance!

[LinasKo]

Hi @ravinpatel 👋

How large are the objects relative to the line? How LineZone works is that it computes notable points on the object and checks that ALL of them cross the line.

Be default, sv.LineZone has a param triggering_anchors=(Position.TOP_LEFT, Position.TOP_RIGHT, Position.BOTTOM_LEFT, Position.BOTTOM_RIGHT).

I'd recommend changing it to triggering_anchors=(Position.CENTER, )

[ravinpatel]

@LinasKo

Here you go https://drive.google.com/file/d/1B4Y912gLkicC6VPo6okMmb3D6OsJYQR3/view?usp=drive_link

Thank you for your help so far, I've been on this for a few days and going crazy so any help is appreciated 🙏

[ravinpatel]

Hi @LinasKo, any chance you could have a look?

[LinasKo]

Unfortunately not. Ping me on Saturday.

Also, please allow access to the video - You should have a request in your gmail.

[ravinpatel]

Okay no problem thank you and I've allowed you access now, let me know if there's anything else I can do to help.

[LinasKo]

I see that, for example, the Cue ball is not annotated at the point where it crosses the line - only a frame or two before, and a frame or two after.

Could you annotate it and check if it then triggers a line crossing?