"The technology of machine vision in robots vacuum is no longer a secret. Ecovacs finally released its first DG70 robots vacuum equipped with AIVI vision technology at the beginning of the year, and Xiaomi's second-generation Mijia robots vacuum (1S) is also engaged. Visual Navigation. Manufacturers have followed suit, carefully matching this "new" technology, or a "new" concept.
In the field of machine vision technology, there are many categories, including structured light and ToF, which have been widely used in mobile phones in recent years, and binocular vision, which is widely used in drones. what is it?
Monocular or binocular?
In the field of machine vision, at present, monocular vision and binocular vision are two types of positioning technologies that have attracted much attention. The huge difference between the two in structure and working principle also leads to different market application status.
Binocular vision positioning refers to the positioning of the three-dimensional spatial position of the target point through the principle of triangulation. The binocular vision positioning algorithm process includes camera calibration, binocular targeting, image processing, feature detection, stereo matching, 3D measurement and attitude measurement.
Due to its high positioning accuracy, low entry threshold, easy 3D composition, and passive light source positioning, binocular vision has many companies in the industry, including Tuyang Technology, Zongmu Technology, and Yunguang Technology.
Still, "in practical applications, binocular vision still has certain limitations," Camsense founder and CEO Christopher told reporters.
As a master tutor of Tsinghua University Shenzhen Graduate School, Christopher has in-depth research in visual human-computer interaction technology. Camsense, the current CEO, mainly provides monocular visual space positioning technology. According to Christopher, he previously founded Taishan Online and has conducted in-depth research on binocular vision technology. For binocular vision, Christopher believes that there are mainly the following three shortcomings:
1. The module is bulky. The positioning accuracy of binocular vision is affected by the baseline (the distance between the two cameras). The longer the baseline, the higher the measurement accuracy. Therefore, binocular vision is less used for positioning in products with high space requirements such as mobile phones;
2. Higher cost. Compared with monocular vision technology, the cost of binocular vision technology is also higher;
3. The effective viewing angle is small. Compared with monocular vision technology, the effective viewing angle of binocular vision will be smaller.
Relatively speaking, Christopher is more optimistic about the application prospects of monocular vision technology.
Monocular vision uses only one camera for localization. There are two main methods of monocular vision positioning: the positioning method based on a single frame image and the positioning method based on two or more frames. Among them, the most widely used are the feature point-based (PnP principle) method based on the single-frame image positioning method, and the IMU inertial sensor-assisted measurement method.
With the gradual maturity of SLAM technology in recent years, the application scenarios of vision technology have become more and more extensive, especially in indoor scenes, monocular vision positioning technology is widely used.
In the field of robots vacuum, the current application of monocular vision technology, including the new robots vacuum products of Ecovacs and Xiaomi mentioned above, has added a visible light camera to assist positioning, and better achieve obstacle avoidance. The iRobot, which has been applied to visual navigation earlier in foreign countries, also uses a monocular camera to continuously track the image during the movement process to estimate the camera pose and the relative position change between the object and the object, thereby establishing a map.
Auxiliary localization of monocular vision
In consumer applications, robots vacuum is increasingly equipped with machine vision; at the same time, in industrial and medical applications, corresponding manufacturers are gradually beginning to try and explore.
Although monocular vision technology has certain advantages in volume and cost, it is not as good as binocular vision in key positioning accuracy. In order to improve the positioning accuracy of monocular vision, auxiliary positioning technology is introduced in practical applications.
The first commonly used auxiliary positioning technology is to add markers to the measured object in a controlled environment.
For industrial inspection, medical and other controllable environments, monocular vision is based on the PnP principle, and the positioning accuracy is improved by adding multiple markers on the object to be measured. For example, in the surgical navigation in the medical field, by adding specific markers to the scalpel used by the doctor, the positioning accuracy can be greatly improved.
The second auxiliary positioning technology is: the introduction of multi-sensor fusion technology and the addition of IMU inertial sensors.
By collecting the acceleration and angular velocity of the IMU inertial sensor (gyroscope, accelerometer, etc.), the motion mode of the measured object is determined, combined with the image collected by the monocular camera, through algorithms such as vision-based feature point matching, sparse optical flow, etc. achieve final positioning. In addition, the introduction of the IMU inertial sensor also makes 3D composition possible through monocular vision technology.
Monocular vision, assisted positioning..., machine vision is so hot, will the lidar, which is the original equipment of robots vacuum, really be gradually replaced?
Before answering this key question, it is necessary to understand the "in-law" relationship between machine vision and lidar.
The "in-law" relationship between machine vision and lidar
Before the rise of SLAM navigation and positioning technology, lidar has been in the orthodox position as the original partner of robots vacuum for a long time. With the rise of visual navigation technology, machine vision has attracted much attention and has become a navigation technology different from lidar.
"In fact, the short-range lidar used in the field of robotics is also done with monocular vision sensors, Christopher told reporters.
In the field of robotics, whether it is vSLAM machine vision or lidar that is currently being talked about, it is actually a machine vision positioning technology. Whether it is the visible light camera in the vSLAM solution or the infrared camera in the lidar solution, there is little difference in essence. However, because lidar uses active light source positioning technology, the positioning is more accurate; the monocular visible light camera uses passive light source for positioning and navigation, and the accuracy is lower. Therefore, it is currently recognized in the industry that lidar is more suitable for positioning and navigation.
In this way, the machine vision and lidar used in robots vacuum are more like a kind of "in-law" relationship. The machine vision that is often mentioned is more of a visible light camera, and the lidar uses an infrared camera.
The difference is that machine vision (visible light camera) is often mentioned. In addition to the positioning and navigation function, it is actually more that it can realize 3D composition and many application expansions brought by image recognition in the future.
Will lidar be replaced by machine vision?
Especially in robots vacuum applications, is this technology iteration or technology fusion?
At present, it belongs to the latter. Including the above-mentioned two major domestic robot vacuum manufacturers, they still use vision as obstacle avoidance to assist the entire navigation system.
At present, 2D vision is still achieved with visible light monocular cameras, but because there is no active light source, the positioning accuracy is poor, and the quality of the constructed maps and trajectory routes is relatively poor. It is easy to implement 3D modeling and can do obstacle avoidance, but when doing SLAM algorithm, due to the narrow horizontal field of view (less than 90 degrees), the efficiency will be much worse.
It can be seen that it will take some years for machine vision to completely replace lidar. Christopher also believes that the development direction of robots vacuum is more of a fusion trend: standard lidar for SLAM algorithm, navigation and positioning, machine vision for obstacle avoidance, 3D vision, deep learning and other corresponding functions.
This also makes the millions of robots vacuum market, the demand for lidar is still considerable. "Our overall business focus this year is in the field of robots vacuum, which is expected to account for more than 90%. Among them, Camsense X1 (lidar) is our main product this year," Christopher told reporters about Camsense's product planning for this year.
X1: Infrared optical spatial positioning algorithm, custom area array infrared sensor
Camsense started the research and development of high-precision positioning algorithms in 2015. The algorithm mentioned here is the Camsense HPP (High Precision Positioning) algorithm based on infrared optical spatial positioning and recognition. According to official information, Camsense HPP can achieve sub-millimeter spatial positioning resolution within a range of 5m x 5m, which is suitable for short-distance precise positioning in indoor environments such as precision equipment measurement and medical surgery navigation.
This algorithm is also the core technology of Camsense.
In addition, Christopher told reporters that Camsense has made many improvements to the light source, lens, and image sensor compared to the LiDAR on the market today. risk.
"Compared to Asia Optical's use of industrial-grade linear image sensors for lidar, Camsense uses a custom area array image sensor." The improvements brought by this include:
Compared with industrial-grade image sensors, custom area array image sensors cost less;
Using an area array image sensor, the vertical field of view is larger than that of a linear image sensor.
According to Christopher, "The vertical field of view of current industrial-grade linear image sensors is generally 0.15 degrees, and the vertical field of view of our custom infrared sensor lidar can reach 40 degrees. This also makes light source calibration and focusing easier in the production process."
LiDAR is the focus of Camsense now, and it is also an important product after it turns to robotics, industrial inspection, and medical applications. Before that, Camsense's monocular vision technology has also been used in TV, VR and other fields.
TV, VR, and then the technical polishing of robots
In 2014, Shenzhen Camsense Co., Ltd. was formally established. At the beginning of its establishment, Camsense positioned traditional visual sensors and first applied it to somatosensory games in the TV field at that time. "At that time, except for TCL, almost all domestic TV manufacturers were our customers, and we provided them with monocular cameras to support somatosensory games in home entertainment."
Why give up the TV market?
Christopher said frankly, "We also know that there is a ceiling in this field, and then we further expanded our market to the VR field that was popular at the time." In 2016, Camsense won the bid of iQiyi for its VR equipment. Provided a spatial positioning system and successfully completed the development of technical products, but because the domestic VR market failed to develop as expected, this product failed to enter mass production.
According to the reporter's understanding, until now, the domestic VR market has been tepid, and well-known devices such as foreign Microsoft HoloLens and HTC Oculus have not appeared.
However, at this stage, Camsense has developed its M-series products, Camsense M1, Camsense M2. Christopher told reporters (public account: reporter) that until now, although these two products for the TV and VR fields are no longer iteratively updated, they are still shipping stably. Up to now, product shipments have reached 150,000 sets.
For technology companies like us, it is very important to choose the field of technology landing. After the technology accumulation in previous years, in 2017, we officially switched our business field to the field of robotics and industrial inspection.
Subsequently, Camsense successively won the bids of COMAC and China Test Group, and developed Camsense M Pro and Camsense S series products for industrial inspection.
"There are many needs for riveting and other stations in the aircraft manufacturing process. The riveting process is realized by the robotic arm, and the robotic arm needs precise guidance and positioning during the movement process. The Camsense S customized solution we provided for COMAC is to carry out Navigation guidance.” Previously, such high-end manufacturing industries were mostly monopolized by foreign countries, and the target of Camsense Camsense S was NDI in Canada. Camsense S can already achieve 0.2mm positioning accuracy.
The next R&D focus of robots vacuum
As mentioned above, in 2017, after entering the second stage, Camsense began to expand its business to the field of robotics. At the same time, the field of robotics is gradually exploring how to better apply machine vision to make up for the existing shortcomings of robots in obstacle avoidance, and try more in the field of image processing.
Christopher believes that the current research and development priorities in the field of robotics, especially robots vacuum, are as follows:
1. Reduce the cost of SLAM navigation and positioning solutions. "Lidar is still the best configuration for SLAM navigation and positioning. How to make the lidar SLAM solution more cost-effective and how to develop better algorithms is a major issue in the current industry."
2. Solve the common obstacle avoidance and object recognition problems. Many robot manufacturers have begun to introduce machine vision to make corresponding solutions;
3. SLAM+ obstacle avoidance fusion scheme. That is, the machine vision + lidar fusion solution, "This is the focus of the current industry, and it is also an important research and development direction for our company."
The fifth year of Camsense: Lidar becomes the focus, robots vacuum becomes the betting field
Camsense is in its fifth year since its founding in 2014.
Technically, it still focuses on monocular vision. Following the joint establishment of a high-precision positioning laboratory with Tsinghua University, an optical laboratory was established in February this year to provide professional R&D and testing sites for optical high-precision positioning systems. + R&D personnel;
In the market, from the initial TV and VR fields to the current industrial, medical, and robotic fields.
In addition, Christopher also emphasized the issue of mass production.
For consumer-grade lidar, mass production is a major difficulty. Camsense has its own trial production workshop, and all products will be trial-produced in the trial production workshop before mass production. Since we built our own trial production workshop in 2018, we have trial-produced 5,000 products. These products are only used for trial production before large-scale mass production, and will not be sold as final products.
According to Christopher, "Camsense will be profitable this year."
In the medical field, Camsense already has intended manufacturers. The main product this year is the Camsense X1 lidar, with a target shipment of more than 500,000 units; the target market is robots vacuum, which is expected to account for more than 90% of the overall business this year.
Christopher told reporters that there are currently three types of customers for Camsense's lidar:
The first category is robots vacuum machine manufacturers, such as 360, which have the ability to do SLAM algorithms;
The second category is solution manufacturers. They make SLAM algorithms based on Camsense's lidar, and then provide complete SLAM solutions to machine manufacturers;
The third category is ODM manufacturers. They make SLAM algorithms based on Camsense's lidar, and then make complete machine solutions for mass production.
"At present, we have more than ten cooperative manufacturers in the three categories."
In the fifth year of Camsense, lidar has become the focus, and robots vacuum has become a betting field. From the perspective of the current situation of robots vacuum in China, machine vision has just emerged, and lidar is still the original equipment, and it is still a key area for the industry to continue to pay attention.
