epic_kitchens.hoa.types

The core set of types that represent hand-object detections

Module Contents

Classes

HandSide	Generic enumeration.
HandState	An enum describing the different states a hand can be in:
FloatVector	A floating-point 2D vector representation
BBox
HandDetection	Dataclass representing a hand detection, consisting of a bounding box,
ObjectDetection	Dataclass representing an object detection, consisting of a bounding box and a
FrameDetections	Dataclass representing hand-object detections for a frame of a video

class epic_kitchens.hoa.types.HandSide

Bases: enum.Enum

Generic enumeration.

Derive from this class to define new enumerations.

LEFT = 0

RIGHT = 1

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

__dir__(self)

Default dir() implementation.

__format__(self, format_spec)

Default object formatter.

__hash__(self)

Return hash(self).

__reduce_ex__(self, proto)

Helper for pickle.

name(self)

The name of the Enum member.

value(self)

The value of the Enum member.

class epic_kitchens.hoa.types.HandState

Bases: enum.Enum

An enum describing the different states a hand can be in: - No contact: The hand isn’t touching anything - Self contact: The hand is touching itself - Another person: The hand is touching another person - Portable object: The hand is in contact with a portable object - Stationary object: The hand is in contact with an immovable/stationary object

NO_CONTACT = 0

SELF_CONTACT = 1

ANOTHER_PERSON = 2

PORTABLE_OBJECT = 3

STATIONARY_OBJECT = 4

__repr__(self)

Return repr(self).

__str__(self)

Return str(self).

__dir__(self)

Default dir() implementation.

__format__(self, format_spec)

Default object formatter.

__hash__(self)

Return hash(self).

__reduce_ex__(self, proto)

Helper for pickle.

name(self)

The name of the Enum member.

value(self)

The value of the Enum member.

class epic_kitchens.hoa.types.FloatVector

A floating-point 2D vector representation

x :np.float32

y :np.float32

to_protobuf(self) → epic_kitchens.hoa.types_pb2.FloatVector

static from_protobuf(vector: epic_kitchens.hoa.types_pb2.FloatVector) → epic_kitchens.hoa.types.FloatVector

__add__(self, other: epic_kitchens.hoa.types.FloatVector) → epic_kitchens.hoa.types.FloatVector

__mul__(self, scaler: float) → epic_kitchens.hoa.types.FloatVector

__iter__(self) → Iterator[float]

property coord(self) → Tuple[float, float]

Return coordinates as a tuple

scale(self, width_factor: float = 1, height_factor: float = 1) → None

Scale x component by width_factor and y component by height_factor

class epic_kitchens.hoa.types.BBox

left :float

top :float

right :float

bottom :float

to_protobuf(self) → epic_kitchens.hoa.types_pb2.BBox

static from_protobuf(bbox: epic_kitchens.hoa.types_pb2.BBox) → epic_kitchens.hoa.types.BBox

property center(self) → Tuple[float, float]

property center_int(self) → Tuple[int, int]

Get center position as a tuple of integers (rounded)

scale(self, width_factor: float = 1, height_factor: float = 1) → None

center_scale(self, width_factor: float = 1, height_factor: float = 1) → None

property coords(self) → Tuple[Tuple[float, float], Tuple[float, float]]

property coords_int(self) → Tuple[Tuple[int, int], Tuple[int, int]]

property width(self) → float

property height(self) → float

property top_left(self) → Tuple[float, float]

property bottom_right(self) → Tuple[float, float]

property top_left_int(self) → Tuple[int, int]

property bottom_right_int(self) → Tuple[int, int]

class epic_kitchens.hoa.types.HandDetection

Dataclass representing a hand detection, consisting of a bounding box, a score (representing the model’s confidence this is a hand), the predicted state of the hand, whether this is a left/right hand, and a predicted offset to the interacted object if the hand is interacting.

bbox :BBox

score :np.float32

state :HandState

side :HandSide

object_offset :FloatVector

to_protobuf(self) → epic_kitchens.hoa.types_pb2.HandDetection

static from_protobuf(detection: epic_kitchens.hoa.types_pb2.HandDetection) → epic_kitchens.hoa.types.HandDetection

scale(self, width_factor: float = 1, height_factor: float = 1) → None

center_scale(self, width_factor: float = 1, height_factor: float = 1) → None

class epic_kitchens.hoa.types.ObjectDetection

Dataclass representing an object detection, consisting of a bounding box and a score (the model’s confidence this is an object)

bbox :BBox

score :np.float32

to_protobuf(self) → epic_kitchens.hoa.types_pb2.ObjectDetection

static from_protobuf(detection: epic_kitchens.hoa.types_pb2.ObjectDetection) → epic_kitchens.hoa.types.ObjectDetection

scale(self, width_factor: float = 1, height_factor: float = 1) → None

center_scale(self, width_factor: float = 1, height_factor: float = 1) → None

class epic_kitchens.hoa.types.FrameDetections

Dataclass representing hand-object detections for a frame of a video

video_id :str

frame_number :int

objects :List[ObjectDetection]

hands :List[HandDetection]

to_protobuf(self) → epic_kitchens.hoa.types_pb2.Detections

static from_protobuf(detections: epic_kitchens.hoa.types_pb2.Detections) → epic_kitchens.hoa.types.FrameDetections

static from_protobuf_str(pb_str: bytes) → epic_kitchens.hoa.types.FrameDetections

get_hand_object_interactions(self, object_threshold: float = 0, hand_threshold: float = 0) → Dict[int, int]

Match the hands to objects based on the hand offset vector that the model uses to predict the location of the interacted object.

Parameters

• object_threshold – Object score threshold above which to consider objects for matching

• hand_threshold – Hand score threshold above which to consider hands for matching.

Returns

A dictionary mapping hand detections to objects by indices

scale(self, width_factor: float = 1, height_factor: float = 1) → None

Scale the coordinates of all the hands/objects. x components are multiplied by the width_factor and y components by the height_factor

center_scale(self, width_factor: float = 1, height_factor: float = 1) → None

Scale all the hands/objects about their center points.