We can imagine a note as a tripel (d, l, o) where d is a document identifier, l a name, and o a list of end character offset tuples. An annotator i contributes to a (multi-) set ai of annotations (tokens). We calculate F1ij = 2 | Ai ∩ Aj | / (| Ai| + | Aj|) for each combination of 2 annotators and ratio of arithmetic mean and standard deviation of F1 over all these combinations (see Hripcsak & Rothschild, 2005). Grouping comments by document or label allows us to calculate F1 per document or label. On this page you will find an overview of the steering functions. Detailed instructions can be found in the brat manual. Comments that have additional or defective parts receive a red "halo" that signals a problem with the note. When you move the mouse over such a note, the detected problems are displayed again. brat eliminates any risk of losing note efforts due to tool drops, forgetting to store work, or even the total failure of an annotator`s computer, by seamlessly communicating all editing operations from annotators to the brat server while they are running. The architecture of the client server brat and the design allow multiple annotators to work simultaneously on the same collection of documents, or even on the same document, and to see the changes of the other as they are made (a certain delay inherent in communication is inherent).
All processing operations are coordinated by the server to ensure that the note remains consistent even if multiple users are simultaneously trying to edit a single note. Some of the many note tasks that can be applied brat are presented on the sample page. (Setting up a brand new brat server requires a few actions, but can be done in as much as five minutes on any system on which a web server is running.) brat also offers seamless integration with the most modern methods of supporting basic annotation such as sentence sharing (English and Japanese) and tokenization (Japanese). All aspects of the note are configured with a simple declarative configuration language. Each document collection has its own configuration, which allows a single brat server to host many projects with different note objectives. Both the brat server and the client implement full Unicode support and thus support nearly 100 different scripts. Notes made in brament can be exported from the interface with a few clicks in a simple distance format that can be easily analyzed, processed and converted to other formats. Note: The gold standard is considered to be the collections/documents from which the comparison is called, while the parallel score provided is considered a set of candidates. Brat visualization is based on Scalable Vector Graphics (SVG), which can be demantablely detailed and accurate. Starting with version 1.3 (Crunchy Frog) provides support for standardization and various functions to link comments to data in external databases, lexical and ontological resources like Freebase, Wikipedia and Open Biomedical Ontologies.
. . .