Fly

An object of the Fly class is an instance of a simulated fly. There can be multiple flies in a Simulation. The Fly object contains parameters related to the fly but not the whole simulation: for example, the set of actuated joints, spawn position, joint actuator parameters (stiffness, damping, etc.), and initial pose.

class flygym.Fly(name: str | None = None, actuated_joints: List = preprogrammed.all_leg_dofs, contact_sensor_placements: List = preprogrammed.all_tarsi_links, xml_variant: str | Path = 'seqik', spawn_pos: Tuple[float, float, float] = (0.0, 0.0, 0.5), spawn_orientation: Tuple[float, float, float] = (0.0, 0.0, np.pi / 2), control: str = 'position', init_pose: str | KinematicPose = 'stretch', floor_collisions: str | List[str] = 'legs', self_collisions: str | List[str] = 'legs', detect_flip: bool = False, joint_stiffness: float = 0.05, joint_damping: float = 0.06, non_actuated_joint_stiffness: float = 1.0, non_actuated_joint_damping: float = 1.0, neck_stiffness: float | None = 10.0, actuator_gain: float | List = 45.0, actuator_forcerange: float | Tuple[float, float] | List = 65.0, tarsus_stiffness: float = 7.5, tarsus_damping: float = 1e-2, friction: float = (1.0, 0.005, 0.0001), contact_solref: Tuple[float, float] = (2e-4, 1e3), contact_solimp: Tuple[float, float, float, float, float] = (0.999, 0.9999, 0.001, 0.5, 2.0), enable_olfaction: bool = False, enable_vision: bool = False, render_raw_vision: bool = False, vision_refresh_rate: int = 500, enable_adhesion: bool = False, adhesion_force: float = 40, draw_adhesion: bool = False, draw_sensor_markers: bool = False, neck_kp: float | None = None, head_stabilization_model: Callable | str | None = None)

A NeuroMechFly environment using MuJoCo as the physics engine.

Attributes:

namestr

The name of the fly model.

actuated_jointsList[str]

List of names of actuated joints.

contact_sensor_placementsList[str]

List of body segments where contact sensors are placed.

spawn_posTuple[float, float, float]

The (x, y, z) position in the arena defining where the fly will be spawn, in mm.

spawn_orientationTuple[float, float, float, float]

The spawn orientation of the fly in the Euler angle format: (x, y, z), where x, y, z define the rotation around x, y and z in radian.

controlstr

The joint controller type. Can be “position”, “velocity”, or “torque”.

init_poseflygym.state.BaseState

Which initial pose to start the simulation from.

floor_collisions :str

Which set of collisions should collide with the floor. Can be “all”, “legs”, “tarsi” or a list of body names.

self_collisionsstr

Which set of collisions should collide with each other. Can be “all”, “legs”, “legs-no-coxa”, “tarsi”, “none”, or a list of body names.

detect_flipbool

If True, the simulation will indicate whether the fly has flipped in the info returned by .step(...). Flip detection is achieved by checking whether the leg tips are free of any contact for a duration defined in the configuration file. Flip detection is disabled for a period of time at the beginning of the simulation as defined in the configuration file. This avoids spurious detection when the fly is not standing reliably on the ground yet.

joint_stiffnessfloat

Stiffness of actuated joints. joint_stiffness : float Stiffness of actuated joints.

joint_dampingfloat

Damping coefficient of actuated joints.

non_actuated_joint_stiffnessfloat

Stiffness of non-actuated joints.

non_actuated_joint_dampingfloat

Damping coefficient of non-actuated joints.

neck_stiffnessUnion[float, None]

Stiffness of the neck joints (joint_Head, joint_Head_roll, and joint_Head_yaw), by default 10.0. The head joints have their stiffness set separately, typically to a higher value than the other non-actuated joints, to ensure that the visual input is not perturbed by unintended passive head movements. If set, this value overrides non_actuated_joint_stiffness.

control: str

The joint controller type. Can be “position”, “velocity”, or “motor”.

tarsus_stiffnessfloat

Stiffness of the passive, compliant tarsus joints.

tarsus_dampingfloat

Damping coefficient of the passive, compliant tarsus joints.

frictionfloat

Sliding, torsional, and rolling friction coefficients.

contact_solref: Tuple[float, float]

MuJoCo contact reference parameters (see MuJoCo documentation for details). Under the default configuration, contacts are very stiff. This is to avoid penetration of the leg tips into the ground when leg adhesion is enabled. The user might want to decrease the stiffness if the stability becomes an issue.

contact_solimp: Tuple[float, float, float, float, float]

MuJoCo contact reference parameters (see MuJoCo docs for details). Under the default configuration, contacts are very stiff. This is to avoid penetration of the leg tips into the ground when leg adhesion is enabled. The user might want to decrease the stiffness if the stability becomes an issue.

enable_olfactionbool

Whether to enable olfaction.

enable_visionbool

Whether to enable vision.

render_raw_visionbool

If enable_vision is True, whether to render the raw vision (raw pixel values before binning by ommatidia).

vision_refresh_rateint

The rate at which the vision sensor is updated, in Hz.

enable_adhesionbool

Whether to enable adhesion.

adhesion_forcefloat

The magnitude of the adhesion force.

draw_adhesionbool

Whether to signal that adhesion is on by changing the color of the concerned leg.

draw_sensor_markersbool

If True, colored spheres will be added to the model to indicate the positions of the cameras (for vision) and odor sensors.

head_stabilization_modelCallable or str optional

If callable, it should be a function that, given the observation, predicts signals that need to be applied to the neck DoFs to stabilizes the head of the fly. If “thorax”, the rotation (roll and pitch) of the thorax is inverted and applied to the head by the neck actuators. If None, no head stabilization is applied.

retinaflygym.vision.Retina

The retina simulation object used to render the fly’s visual inputs.

arena_root = dm_control.mjcf.RootElement

The root element of the arena.

action_spacegymnasium.core.ObsType

Definition of the fly’s action space.

observation_spacegymnasium.core.ObsType

Definition of the fly’s observation space.

modeldm_control.mjcf.RootElement

The MuJoCo model.

vision_update_masknp.ndarray

The refresh frequency of the visual system is often loser than the same as the physics simulation time step.

change_segment_color(physics: dm_control.mjcf.Physics, segment: str, color)

Change the color of a segment of the fly.

Parameters:

physicsmjcf.Physics

The physics object of the simulation.

segmentstr

The name of the segment to change the color of.

colorTuple[float, float, float, float]

Target color as RGBA values normalized to [0, 1].

close()

Release resources allocated by the environment.

get_info()

Any additional information that is not part of the observation. This method always returns an empty dictionary unless extended by the user.

Returns:

Dict[str, Any]

The dictionary containing additional information.

get_observation(sim: Simulation) → ObsType

Get observation without stepping the physics simulation.

Returns:

ObsType

The observation as defined by the environment.

get_reward()

Get the reward for the current state of the environment. This method always returns 0 unless extended by the user.

Returns:

float

The reward.

init_floor_contacts(arena: BaseArena)

Initialize contacts between the fly and the floor. This is called by the Simulation after the fly is placed in the arena and before setting up the physics engine.

Parameters:

arenaBaseArena

The arena in which the fly is placed.

is_terminated()

Whether the episode has terminated due to factors that are defined within the Markov Decision Process (e.g. task completion/ failure, etc.). This method always returns False unless extended by the user.

Returns:

bool

Whether the simulation is terminated.

is_truncated()

Whether the episode has terminated due to factors beyond the

Markov Decision Process (e.g. time limit, etc.). This method always returns False unless extended by the user.

Returns:

bool

Whether the simulation is truncated.

post_init(sim: Simulation)

Initialize attributes that depend on the arena or physics of the simulation.

Parameters:

arenaBaseArena

The arena in which the fly is placed.

physicsmjcf.Physics

The physics object of the simulation.

update_colors(physics: dm_control.mjcf.Physics)

Update the colors of the fly’s body segments. This is typically called by Simulation.render to update the colors of the fly before the cameras do the rendering.

Parameters:

physicsmjcf.Physics

The physics object of the simulation.

property vision_update_mask: ndarray

The refresh frequency of the visual system is often loser than the same as the physics simulation time step. This 1D mask, whose size is the same as the number of simulation time steps, indicates in which time steps the visual inputs have been refreshed. In other words, the visual input frames where this mask is False are repetitions of the previous updated visual input frames.