Customer case study

Could you tell us about your company’s business?

SkymatiX is a company with strengths in remote sensing and GIS. Remote sensing is a technology that allows us to sense things without touching them. For example, we observe the ground from a distance from the sky, collect image-based information, and carry out various analyses using this information.

We also work with geographic information systems (GIS), adding depth to various information on maps, and thereby adding value to nature. Usually, companies are strong in one or the other of these areas, but we feel that SkymatiX is different in that it has strengths in both.

What kind of data creation were you doing before using our service?

We used the services of a number of similar annotation (*1) companies. For services where Japanese people do the annotation work, the cost of each item is high, so we often outsourced to overseas vendors. Of course, when the volume was small, we would do the work in-house. Our decision about whether or not to outsource was based on volume and cost.

How much volume determines whether you do the work in house or outsource it?

Any volume that we do not feel like handling ourselves. That volume depends on the onerousness of the work , but the task we requested from your company this time was a relatively burdensome task, as annotating satellite image data even for a single image is a challenge. In such cases, we make the decision to outsource, even if it’s just five satellite images. On the other hand, for simple tasks such as marking targets, we often do such tasks in-house if the volume is a hundred items or so.

Delivery deadlines are another factor. Even if we were thinking of outsourcing a job due to a tight deadline, it would still take time to communicate to get the details just right regardless of the rush we were in, so if the volume is low, we may just do the work ourselves.—The primary decision factor is whether the task would be painful to do in house.

Outsourcing does involve work such as creating a procedure manual, holding meetings, and checking the deliverables, so our decision as to whether to outsource is taken by comparing which way is more challenging.

You mentioned that you have used overseas annotation vendors. Are there any hurdles in doing so?

Not particularly. In our case, for example, we use many images of vegetables and the ground, and we don’t use faces of persons that could be linked to personal information, so using overseas vendors is not a problem. However, if we sign an NDA and need to get a job done that involves photos taken inside a particular factory, it would be difficult or even impossible to send these photos overseas. There have been times when we decided to do the job in house in such cases.

In that respect, we are grateful that YAZAKI’s service gives us peace of mind in terms of security and compliance management system.

Have there been any issues or bottlenecks when you requested work from other annotation vendors?

Yes indeed. When we outsource, we are the ones who ultimately use the delivered products, so the quality of the delivered items is quite important. In some cases, a lot of time is required to perform checks. We often need to ask for revisions.

What kinds of items did you feel required many revisions?

Satellite images and aerial photos, for example. Simple tasks can be extremely simple. For example, if the task is just to put a circle around cabbages , issuing the instructions is easy for the requester and understanding them is easy for the receiver.

The job we requested from YAZAKI this time included complex tasks such as indicating the boundaries of rivers . We could have created a procedure manual in advance, but there were many parts that would remain unclear until actually done. In that sense, we think the task was a challenging one.

Among the tasks we gave you, we predicted that the annotation of trees would likely be the most difficult. The colors of grass and trees are the same when viewed from above, so the difference lies in the shadows created by the height of the trees. Therefore, we gave the order, “Please find those shadows and annotate them,” but this task must have been quite difficult.

Is it very important to define the requirements beforehand?

Yes, it is. Before performing the overall annotation task, you asked us about the requirements and we believe that this was beneficial in terms of saving time during the subsequent processes.

Other companies offer services that they tout as “fast” or “cheap,” but when you check, they turn out to be rush jobs. For example, if we give out a job consisting of 10 images, our inspection of the delivered items may find that the entire job needs to be redone, or that many revisions are required, and we sometimes conclude that it would be faster for us to do the job ourselves.

Annotation companies offer “cheap and fast services” as well as services that cost a little more but may be worth the price when considering the effort they save later, and deciding which is most advantageous can be a dilemma. For jobs that must be done carefully, YAZAKI is certainly a prime candidate as we consider which service to use.

Could you tell us what made you decide to use our service?

Certainly. Initially, you approached us saying, “We are confident in our quality,” so we thought we’d give you a try. We received a large project just around that time, and we wondered if our current annotation companies alone would be able to handle it, so we decided to employ various companies.

Could you tell us what you liked about using our service?

First and foremost, the fact that you created good annotation data without much effort on our part. We think this was because you took the time to define the various requirements at the beginning. As you duly clarified the requirements, we could just ask you to do the remainder in a similar manner, so no effort was required on our part. The quality of the data that you delivered to us was top level among the various participating companies.

To ensure accuracy, it is better to have engineers on the task to some extent, but is striking some kind of balance important?

Yes, it is. If you consider the time and effort it takes to create a procedure manual and explain things to others so people are clear on the kind of annotation you want, you might decide that it’s faster to do it yourself if it’s only about 100 images. On the other hand, a volume of 10,000 images is a tough challenge and there are times when we will outsource the work even if we have to spend time and effort to explain the job to others. Whether we feel up to taking up the challenge of a big job on our own is also part of the equation.

Documentation skills for procedure manuals are also important.(*2) If you produce a procedure manual that’s difficult to understand, the outsourcee may have trouble understanding it, resulting in a large number of inquiries, and the delivered items might have problems. The time and effort spent on such things are also a concern.

How did you find the proposal ability of our reps and their skill in defining requirements prior to the work?

They were more than satisfactory. When you told us that YAZAKI does annotation, we hesitated as to whether or not we should ask you because the types of sample images you provided were different from satellite images.

When we asked you whether you would be able to handle such images, you told us you could, and given the large volume at that time, we considered giving you a try. We were also pleased with the content of your proposal, which was in line with our development requirements.

Please tell us if there is anything you expect from our company in the future.

There are two types of annotation vendors: companies that say, “We will accept the job if it’s standard,” and companies that say, “The requirements are complex and the patterns are unusual, but we’ll give it a try.” YAZAKI fits the latter type as a company that is willing to look at what’s on the table and we believe that you have the capacity to take on such challenges.

Going forward, we expect higher technical skills. We may request difficult annotations, for example, images where it is difficult to identify outlines, or images that are highly confidential, and we look forward to your cooperation at such times.

*1. Annotation: The process of giving meaning to (tagging) data used for teaching AI

*2. We also support the creation of procedure manuals.