Glossary of terms for crowdsourcing.txt

+===============+=====================+==========================================================+
| UHRS Term     | MTurk equivalent    | Description                                              |
+===============+=====================+==========================================================+
| HIT           | HIT                 | Human Intelligence Task is a single example for          |
|               |                     | evaluation (e.g., a single pair of images to evaluate).  |
+---------------+---------------------+----------------------------------------------------------+
| Task          | Task                | A collection of HITs (e.g., a set of 30 image pairs to   |
|               |                     | evaluate).                                               |
+---------------+---------------------+----------------------------------------------------------+
| Task Group    | Batch               | A collection of tasks that have the same rules /         |
|               |                     | judgment groups.                                         |
+---------------+---------------------+----------------------------------------------------------+
| HITApp        | HIT Type /          | The application on which the HIT is run. It defines the  |
|               | HIT Template        | structure, interface, and settings. It includes          |
|               |                     | parameters such as the reward per HIT, worker            |
|               |                     | qualifications, and time limits.                         |
+---------------+---------------------+----------------------------------------------------------+
| Judge         | Worker / Turker     | A person who completes HITs on the platform in exchange  |
|               |                     | for payment.                                             |
+---------------+---------------------+----------------------------------------------------------+
| Judgment      | Assignment          | The answer of a single judge to a HIT. A specific        |
|               |                     | instance of a HIT completed by a worker. Multiple        |
|               |                     | workers can complete the same HIT for redundancy and     |
|               |                     | quality control.                                         |
+---------------+---------------------+----------------------------------------------------------+
| Vendor and    | Requester           | Vendor in UHRS supplies tasks to the platform. Client    |
| Client        |                     |  is the organization or individual requesting judgments. |
+---------------+---------------------+----------------------------------------------------------+
| Normal Task   | Standard HIT        | A task that workers complete without predefined “gold”   |
|               |                     | answers.                                                 |
+---------------+---------------------+----------------------------------------------------------+
| Spam Task     | Gold Standard HIT / | A task with known correct answers to evaluate worker     |
|               | Honeypot Question   | accuracy and detect spammers.                            |
+---------------+---------------------+----------------------------------------------------------+
| Qualification | Qualification Test  | A task with known answers used to assess workers before  |
| Task          |                     | they can work on production tasks.                       |
+---------------+---------------------+----------------------------------------------------------+
| RTA Task      | Training HIT        | A task where workers receive immediate feedback after    |
|               |                     | submission to improve accuracy.                          |
+---------------+---------------------+----------------------------------------------------------+
| Crowd Judge   | MTurk Worker        | Independent workers who sign up to complete tasks (HITs) |
|               | (Turker)            | and are paid per task completed.                         |
+---------------+---------------------+----------------------------------------------------------+
| Crowd Judge   | N/A                 | MTurk does not have direct vendors supplying workers.    |
| Vendor        |                     | Instead, any worker on the platform can complete tasks.  |
+---------------+---------------------+----------------------------------------------------------+
| Managed Judge | Private Workforce / | In some cases, Requesters (task creators) may recruit    |
|               | Managed Workers     | specific workers or use external workforce providers to  |
|               |                     | complete tasks at an hourly rate.                        |
+---------------+---------------------+----------------------------------------------------------+
| Managed Judge | Third-Party         | Companies can use third-party services to supply managed |
| Vendor        | Workforce Providers | workers, but this is not built into MTurk itself.        |
+---------------+---------------------+----------------------------------------------------------+
| Judge Group   | Qualification Group | Requesters can create a Qualification Group to control   |
|               |                     | which workers can access specific HITs based on          |     
|               |                     | predefined criteria.                                     |
+---------------+---------------------+----------------------------------------------------------+