Use the Vision Set with Blockly

Blockly’s blocks have been designed to be used with the Vision Set. It keeps Ned’s Ecosystem easy to use with Niryo Studio.

In this section, we will see how to use the Vision Set with these blocks.

Blocks Description

Color Block

Allows the user to choose a color among the list of available colors.

This color will be used in the Vision Pick Block and the Is detected block.

Shape Block

Allows the user to choose a shape among the list of available shapes.

The selected shape will be used in the Vision Pick Block and the Is detected block.

Vision Pick Block

This block will make Ned pick an object matching the Color Block and Shape Block.

The picking process is:

1. Opening the gripper

2. Going over the object (5 cm)

3. Going down

4. Closing the gripper

5. Going back to 2.

The pick will happen in the selected workspace (default_workspace in the example)

The height offset is the height at which the end effector will come to take the object, based on the workspace height.

Examples for height offset:

• With the Vacuum Pump equipped, you can use the height of the object.

• With the Standard Gripper equipped, it can be half the height of the object to take it from its middle.

This block acts as a condition and so, returns a Boolean.

Vision Pick with Observation Pose Block

Equivalent to a Move Pose to the Observation Pose followed by a vision pick.

Is detected block

Represents whether an object is matching Color Block and Shape Block is present in the workspace.

This block returns a Boolean.

Examples

Please find below some use cases in Blockly. For each example, the sequence will be detailed step by step and you will have to set an observation position.

Important

In order to realise these examples, you need to have your robot calibrated, and its tool plugged & scanned.

Moreover, you will need to register your workspace. In the following examples, the name used is default_workspace but feel free to chose yours!

Simple Vision pick and place

Task

Ned has to pick an object from its workspace using vision. Then, it should place it anywhere.

Solution & explanation

To pick an object using Vision, you can use the Vision Pick Block. It will return True if an object is detected and picked, so it has to be put in a logic condition ⇨ if the action happens, we can place the picked object.

You can add an action in case the picking did not happen with else. In this case, the robot will turn on the learning mode.

We need to see all 4 landmarks when the Vision Pick is called. In order to do that, we have to move over the workspace. We can do that by using a Move Pose block, but for the sake of this User Guide, we use the Vision Pick with Observation Pose Block

So, a complete pick and place process could be resumed in this easy sequence, which defines an observation position and picks an object if there is a matching object in the workspace to place it elsewhere, or activates learning mode if no object matches the chosen attributes.

Empty the Workspace with Vision pick and place

Task 1

Ned has to pick any object in the workspace then place them all on the same spot.

The workspace can have any number of object without fixed color/shape.

Solution & explanation 1

To make the Vision pick task, you can use the Vision Pick with Observation Pose Block which will return True each time the action happens. By using a loop as a condition (here, repeat while), the robot will continue the process until it picked all the objects of the workspace.

Then, a Place from Pose Block is added in order to place each picked object.

In this sequence, Ned will pick all the objects it can detect on the workspace then activate its learning mode.

Multi-reference packaging with Vision pick & place

Task 2

Ned has to take all the objects from the workspace and place them at different positions depending on their attributes ⇨ the red objects will be placed in an area and the other ones in a different area.

Solution & explanation 2

You will start the process by using the Vision Pick with Observation Pose Block to interact with the red objects only. If the action happens, the object will be placed on a certain pose.

Then, a else if condition is added. That will be triggered if the first vision pick did not happen, which means that there are no red objects anymore.

With this program, Ned will try to pick a red object, then place it in an area. If there is no red object in the workspace, Ned will take another object to place it in another area.

Now, we can use a loop to repeat this sequence for a given amount of objects using variables.

Vision pick with the Conveyor Belt

Task 3

Ned has to take a number of objects from a workspace position on a Conveyor Belt, then place it in another area.

Solution & explanation 3

With Blockly, Ned is not able to pick a moving object, but it can control the Conveyor Belt! So, we can do:

1. Move Ned to an observation position above the Conveyor Belt with Move pose

2. Turn on the Conveyor Belt with Control Conveyor

3. Wait to see an object entering the workspace with Repeat until object detected

4. Stop the Conveyor Belt with Stop Conveyor

5. Pick the object and place it in the position of your choice with Vision Pick Block