Legacy API

Warning

This API, inherited from the 0.x.x versions, is deprecated and will be removed in a future release. Please use the new API (the rest of the documentation) instead.

class flygym.core.NeuroMechFly(sim_params: Parameters = None, actuated_joints: list = preprogrammed.all_leg_dofs, contact_sensor_placements: list = preprogrammed.all_tarsi_links, output_dir: Path | None = None, arena: BaseArena = None, xml_variant: str | Path = 'seqik', spawn_pos: tuple[float, float, float] = (0.0, 0.0, 0.5), spawn_orientation: tuple[float, float, float] = (0.0, 0.0, np.pi / 2), control: str = 'position', init_pose: str | KinematicPose = 'stretch', floor_collisions: str | list[str] = 'legs', self_collisions: str | list[str] = 'legs', detect_flip: bool = False)

Bases: SingleFlySimulation

A NeuroMechFly environment using MuJoCo as the physics engine. This class is a wrapper around the SingleFlySimulation and is provided for backward compatibility.

Parameters:

sim_paramsflygym.Parameters

Parameters of the MuJoCo simulation.

actuated_jointslist[str], optional

List of names of actuated joints. By default all active leg DoFs.

contact_sensor_placementslist[str], optional

List of body segments where contact sensors are placed. By default all tarsus segments.

output_dirPath, optional

Directory to save simulation data. If None, no data will be saved. By default None.

arenaflygym.arena.BaseArena, optional

The arena in which the fly is placed. FlatTerrain will be used if not specified.

xml_variant: str or Path, optional

The variant of the fly model to use. Multiple variants exist because when replaying experimentally recorded behavior, the ordering of DoF angles in multi-DoF joints depends on how they are configured in the upstream inverse kinematics program. Two variants are provided: “seqik” (default) and “deepfly3d” (for legacy data produced by DeepFly3D, Gunel et al., eLife, 2019). The ordering of DoFs can be seen from the XML files under flygym/data/mjcf/.

spawn_postuple[float, float, float], optional

The (x, y, z) position in the arena defining where the fly will be spawn, in mm. By default (0, 0, 0.5).

spawn_orientationtuple[float, float, float], optional

The spawn orientation of the fly in the Euler angle format: (x, y, z), where x, y, z define the rotation around x, y and z in radian. By default (0.0, 0.0, pi/2), which leads to a position facing the positive direction of the x-axis. Important: the definition of the spawn orientation has changed since the v1.1.0. Please use the new API (flygym.Simulation and flygym.Fly). See documentation and tutorials at https://neuromechfly.org/.

controlstr, optional

The joint controller type. Can be “position”, “velocity”, or “torque”, by default “position”.

init_poseBaseState, optional

Which initial pose to start the simulation from. By default “stretch” kinematic pose with all legs fully stretched.

floor_collisions :str

Which set of collisions should collide with the floor. Can be “all”, “legs”, “tarsi” or a list of body names. By default “legs”.

self_collisionsstr

Which set of collisions should collide with each other. Can be “all”, “legs”, “legs-no-coxa”, “tarsi”, “none”, or a list of body names. By default “legs”.

detect_flipbool

If True, the simulation will indicate whether the fly has flipped in the info returned by .step(...). Flip detection is achieved by checking whether the leg tips are free of any contact for a duration defined in the configuration file. Flip detection is disabled for a period of time at the beginning of the simulation as defined in the configuration file. This avoids spurious detection when the fly is not standing reliably on the ground yet. By default False.

Attributes:

sim_paramsflygym.Parameters

Parameters of the MuJoCo simulation.

timestep: float

Simulation timestep in seconds.

output_dirPath

Directory to save simulation data.

arenaflygym.arena.BaseArena

The arena in which the fly is placed.

spawn_postuple[float, float, float]

The (x, y, z) position in the arena defining where the fly will be spawn, in mm.

spawn_orientationtuple[float, float, float, float]

The spawn orientation of the fly in the Euler angle format: (x, y, z), where x, y, z define the rotation around x, y and z in radian.

controlstr

The joint controller type. Can be “position”, “velocity”, or “torque”.

init_poseflygym.state.BaseState

Which initial pose to start the simulation from.

render_modestr

The rendering mode. Can be “saved” or “headless”.

actuated_jointslist[str]

List of names of actuated joints.

contact_sensor_placementslist[str]

List of body segments where contact sensors are placed. By default all tarsus segments.

detect_flipbool

If True, the simulation will indicate whether the fly has flipped in the info returned by .step(...). Flip detection is achieved by checking whether the leg tips are free of any contact for a duration defined in the configuration file. Flip detection is disabled for a period of time at the beginning of the simulation as defined in the configuration file. This avoids spurious detection when the fly is not standing reliably on the ground yet.

retinaflygym.vision.Retina

The retina simulation object used to render the fly’s visual inputs.

arena_root = dm_control.mjcf.RootElement

The root element of the arena.

physics: dm_control.mjcf.Physics

The MuJoCo Physics object built from the arena’s MJCF model with the fly in it.

curr_timefloat

The (simulated) time elapsed since the last reset (in seconds).

action_spacegymnasium.core.ObsType

Definition of the simulation’s action space as a Gym environment.

observation_spacegymnasium.core.ObsType

Definition of the simulation’s observation space as a Gym environment.

modeldm_control.mjcf.RootElement

The MuJoCo model.

vision_update_masknp.ndarray

The refresh frequency of the visual system is often loser than the same as the physics simulation time step. This 1D mask, whose size is the same as the number of simulation time steps, indicates in which time steps the visual inputs have been refreshed. In other words, the visual input frames where this mask is False are repetitions of the previous updated visual input frames.

render()

Compute the render frames as specified by render_mode during the initialization of the environment.

The environment’s metadata render modes (env.metadata[“render_modes”]) should contain the possible ways to implement the render modes. In addition, list versions for most render modes is achieved through gymnasium.make which automatically applies a wrapper to collect rendered frames.

Note:

As the render_mode is known during __init__, the objects used to render the environment state should be initialised in __init__.

By convention, if the render_mode is:

• None (default): no render is computed.

• “human”: The environment is continuously rendered in the current display or terminal, usually for human consumption. This rendering should occur during step() and render() doesn’t need to be called. Returns None.

• “rgb_array”: Return a single frame representing the current state of the environment. A frame is a np.ndarray with shape (x, y, 3) representing RGB values for an x-by-y pixel image.

• “ansi”: Return a strings (str) or StringIO.StringIO containing a terminal-style text representation for each time step. The text can include newlines and ANSI escape sequences (e.g. for colors).

• “rgb_array_list” and “ansi_list”: List based version of render modes are possible (except Human) through the wrapper, gymnasium.wrappers.RenderCollection that is automatically applied during gymnasium.make(..., render_mode="rgb_array_list"). The frames collected are popped after render() is called or reset().

Note:

Make sure that your class’s metadata "render_modes" key includes the list of supported modes.

Changed in version 0.25.0: The render function was changed to no longer accept parameters, rather these parameters should be specified in the environment initialised, i.e., gymnasium.make("CartPole-v1", render_mode="human")

save_video(path: str | Path, stabilization_time=0.02)

class flygym.core.Parameters(timestep: float = 0.0001, joint_stiffness: float = 0.05, joint_damping: float = 0.06, non_actuated_joint_stiffness: float = 1.0, non_actuated_joint_damping: float = 1.0, actuator_gain: float = 40.0, tarsus_stiffness: float = 10.0, tarsus_damping: float = 10.0, friction: float = (1.0, 0.005, 0.0001), gravity: tuple[float, float, float] = (0.0, 0.0, -9810.0), contact_solref: tuple[float, float] = (0.0002, 1000.0), contact_solimp: tuple[float, float, float, float, float] = (0.999, 0.9999, 0.001, 0.5, 2.0), enable_olfaction: bool = False, enable_vision: bool = False, render_raw_vision: bool = False, render_mode: str = 'saved', render_window_size: tuple[int, int] = (640, 480), render_playspeed: float = 0.2, render_fps: int = 30, render_camera: str = 'Animat/camera_left', render_timestamp_text: bool = False, render_playspeed_text: bool = True, vision_refresh_rate: int = 500, enable_adhesion: bool = False, adhesion_force: float = 40, draw_adhesion: bool = False, draw_sensor_markers: bool = False, draw_contacts: bool = False, decompose_contacts: bool = True, force_arrow_scaling: float = nan, tip_length: float = 10.0, contact_threshold: float = 0.1, draw_gravity: bool = False, gravity_arrow_scaling: float = 0.0001, align_camera_with_gravity: bool = False, camera_follows_fly_orientation: bool = False, neck_kp: float | None = None)

Bases: object

Parameters of the MuJoCo simulation.

Attributes:

timestepfloat

Simulation timestep in seconds, by default 0.0001.

joint_stiffnessfloat

Stiffness of actuated joints, by default 0.05.

joint_dampingfloat

Damping coefficient of actuated joints, by default 0.06.

non_actuated_joint_stiffnessfloat

Stiffness of non-actuated joints, by default 1.0. (made stiff for better stability)

non_actuated_joint_dampingfloat

Damping coefficient of non-actuated joints, by default 1.0. (made stiff for better stability)

actuator_gainfloat

if control is “position”, this is the position gain of the position actuators. If control is “velocity”, this is the velocity gain of the velocity actuators. If control is “torque”, this is ignored. By default 40.0.

tarsus_stiffnessfloat

Stiffness of the passive, compliant tarsus joints, by default 2.2.

tarsus_dampingfloat

Damping coefficient of the passive, compliant tarsus joints, by default 0.126.

frictionfloat

Sliding, torsional, and rolling friction coefficients, by default (1, 0.005, 0.0001)

gravitytuple[float, float, float]

Gravity in (x, y, z) axes, by default (0., 0., -9.81e3). Note that the gravity is -9.81 * 1000 due to the scaling of the model.

contact_solref: tuple[float, float]

MuJoCo contact reference parameters (see MuJoCo documentation for details). By default (9.99e-01, 9.999e-01, 1.0e-03, 5.0e-01, 2.0e+00). Under the default configuration, contacts are very stiff. This is to avoid penetration of the leg tips into the ground when leg adhesion is enabled. The user might want to decrease the stiffness if the stability becomes an issue.

contact_solimp: tuple[float, float, float, float, float]

MuJoCo contact reference parameters (see MuJoCo docs for details). By default (9.99e-01, 9.999e-01, 1.0e-03, 5.0e-01, 2.0e+00). Under the default configuration, contacts are very stiff. This is to avoid penetration of the leg tips into the ground when leg adhesion is enabled. The user might want to decrease the stiffness if the stability becomes an issue.

enable_olfactionbool

Whether to enable olfaction, by default False.

enable_visionbool

Whether to enable vision, by default False.

render_raw_visionbool

If enable_vision is True, whether to render the raw vision (raw pixel values before binning by ommatidia), by default False.

render_modestr

The rendering mode. Can be “saved” or “headless”, by default “saved”.

render_window_sizetuple[int, int]

Size of the rendered images in pixels, by default (640, 480).

render_playspeedfloat

Play speed of the rendered video, by default 0.2.

render_fpsint

FPS of the rendered video when played at render_playspeed, by default 30.

render_camerastr

The camera that will be used for rendering, by default “Animat/camera_left”.

render_timestamp_textbool

If True, text indicating the current simulation time will be added to the rendered video.

render_playspeed_textbool

If True, text indicating the play speed will be added to the rendered video.

vision_refresh_rateint

The rate at which the vision sensor is updated, in Hz, by default 500.

enable_adhesionbool

Whether to enable adhesion. By default False.

draw_adhesionbool

Whether to signal that adhesion is on by changing the color of the concerned leg. By default False.

adhesion_forcefloat

The magnitude of the adhesion force. By default 20.

draw_sensor_markersbool

If True, colored spheres will be added to the model to indicate the positions of the cameras (for vision) and odor sensors. By default False.

draw_contactsbool

If True, arrows will be drawn to indicate contact forces between the legs and the ground. By default False.

decompose_contactsbool

If True, the arrows visualizing contact forces will be decomposed into x-y-z components. By default True.

force_arrow_scalingfloat

Scaling factor determining the length of arrows visualizing contact forces. By default 1.0.

tip_lengthfloat

Size of the arrows indicating the contact forces in pixels. By default 10.

contact_thresholdfloat

The threshold for contact detection in mN (forces below this magnitude will be ignored). By default 0.1.

draw_gravitybool

If True, an arrow will be drawn indicating the direction of gravity. This is useful during climbing simulations. By default False.

gravity_arrow_scalingfloat

Scaling factor determining the size of the arrow indicating gravity. By default 0.0001.

align_camera_with_gravitybool

If True, the camera will be rotated such that gravity points down. This is useful during climbing simulations. By default False.

camera_follows_fly_orientationbool

If True, the camera will be rotated so that it aligns with the fly’s orientation. By default False.

perspective_arrow_lengthbool

If true, the length of the arrows indicating the contact forces will be determined by the perspective.

neck_kpfloat, optional

Position gain of the neck position actuators. If supplied, this will overwrite actuator_gain for the neck actuators. Otherwise, actuator_gain will be used.

actuator_gain: float = 40.0

adhesion_force: float = 40

align_camera_with_gravity: bool = False

camera_follows_fly_orientation: bool = False

contact_solimp: tuple[float, float, float, float, float] = (0.999, 0.9999, 0.001, 0.5, 2.0)

contact_solref: tuple[float, float] = (0.0002, 1000.0)

contact_threshold: float = 0.1

decompose_contacts: bool = True

draw_adhesion: bool = False

draw_contacts: bool = False

draw_gravity: bool = False

draw_sensor_markers: bool = False

enable_adhesion: bool = False

enable_olfaction: bool = False

enable_vision: bool = False

force_arrow_scaling: float = nan

friction: float = (1.0, 0.005, 0.0001)

gravity: tuple[float, float, float] = (0.0, 0.0, -9810.0)

gravity_arrow_scaling: float = 0.0001

joint_damping: float = 0.06

joint_stiffness: float = 0.05

neck_kp: float | None = None

non_actuated_joint_damping: float = 1.0

non_actuated_joint_stiffness: float = 1.0

perspective_arrow_length = False

render_camera: str = 'Animat/camera_left'

render_fps: int = 30

render_mode: str = 'saved'

render_playspeed: float = 0.2

render_playspeed_text: bool = True

render_raw_vision: bool = False

render_timestamp_text: bool = False

render_window_size: tuple[int, int] = (640, 480)

tarsus_damping: float = 10.0

tarsus_stiffness: float = 10.0

timestep: float = 0.0001

tip_length: float = 10.0

vision_refresh_rate: int = 500