====== Search ====== .. note:: This chapter describes the search behavior introduced by the relevance redesign (design document ``doc/search-relevance.md``, decision D-12). It is the normative target; the behavior stated here is pinned by automated tests — both supported engines (Elasticsearch and embedded Lucene) must satisfy it identically. The behavior below is implemented; only the relevance tuning on production data is still ongoing. Search behavior =============== What a user may rely on, independent of the configured engine: Multiple words Every entered word must occur somewhere in the record (logical AND). Word order never decides *whether* a record matches — only how high it ranks. If no record matches all words, the portal automatically retries with "any of the words" and clearly labels the result as such. Case and diacritics Matching ignores letter case and diacritics: ``rehor`` finds ``Řehoř``. Diacritics still matter for **ranking** — a user who types ``Řehoř`` sees the exact-diacritics match first. Phrases Words enclosed in double quotes (``"kronika města"``) must occur as an exact phrase. A phrase never matches across values of two different description items. An unbalanced quote has no special meaning. Partial words In queries of up to six words, partially typed words match automatically: a fragment of at least three letters (``relevance.partialMinLength``) matches *anywhere inside* a word — "pardub" and "ardub" both find Pardubice, "univ bratisl" finds both "Univerzita Bratislava" and "Bratislavská univerzita". Shorter fragments must match a whole word, and fragments never span word boundaries. Ranking is unaffected by the extra recall: records matching the words exactly rank above word-start matches, which rank above mid-word matches. There are no query operators besides quotes — an asterisk is a literal character with no meaning. (Substring matching is served by trigram companion fields; it costs index size, not query time.) A longer query is treated as pasted text — a citation, a copied sentence — whose words are complete and match whole; if a word or two miss, the automatic any-word retry (see *Multiple words* above) still finds the record. Inflected words Where the language of the described material (``search.content-locale``) has a stemmer — Czech does — a word matches its inflected forms in both directions: ``hrad`` finds records saying ``hradu`` or ``hradem``, and ``hradu`` finds ``hrad``. Quoted phrases are exempt: they require the exact words. Like partial matching, this only widens what is found — records carrying the exact typed form always rank above those matching an inflected form only. A light stemmer handles regular declension; forms that change inside the word (``dům``/``domu``) remain distinct words. It can be switched off with ``relevance.stemming: false`` (query-side only, no reindex needed). Stop words Common stop words of the described material's language ("v", "a", "na", … in Czech) never cause empty results; a query consisting only of stop words is still answered. The list follows ``search.content-locale`` and is the same on both engines. Supported: Czech, Danish, Dutch, English, French, German, Hungarian, Italian, Norwegian, Russian, Spanish and Swedish; any other language removes nothing, which only makes matching stricter. What is searched All indexed description items participate in the fulltext search unless excluded by ``fulltext: false`` in ``types.yaml`` — names, descriptions, reference labels, numbers, and the boundary years of datings. Note that only the *boundary* years of a dating are text-searchable ("1945" does not find a record dated 1940–1950); searching inside date ranges is what the dating facet is for. Facets of the general search Searching without choosing a section spans every record type, so the section facets of ``searchConfig.yaml`` cannot apply — their item types belong to a section. That search therefore offers three facets that need **no configuration at all**: the record type, the record's dating and "related to". They are part of the application, they appear in every deployment, and they cannot be renamed or switched off. "Related to" lets the reader name the record a result must be related to — an access point, an archive, an archival fonds or a finding aid, the things a relation actually points at. Archival records and collections are not offered: a relation to one particular record is not something readers look for. A relation set from a record's own page is still honoured whatever the record type, so a link from anywhere in the portal keeps working. It searches records the way the search box does, so the rule under **Partial words** applies: a fragment of three letters or more matches inside a word, a shorter one has to match a whole word. A two-letter place name is therefore found by typing it out ("Aš" finds Aš), but no shorter fragment will narrow towards it — which is why the list simply stays empty below three characters instead of announcing that nothing matches. The dating facet there covers the record's dating whatever item type records it, so it also finds records dated only under, say ``DATE_OF_CONTENT``. A record with several datings matches when any one of them falls in the selected period — and only then: a record dated 1850–1860 and again in 1600 is not offered for 1700, which lies between two of its datings rather than within either. A record with no dating at all is **not** matched by a dating filter — it is not in 1805–1852. Where such records are among the results, the filter offers to include them and says how many there are; where every record is dated, as in an archival description, nothing is shown. This needs no configuration either. Result ordering =============== .. list-table:: :widths: 22 78 :header-rows: 1 * - Mode - Meaning * - relevance - Best match first (see `How ranking works`_); the default whenever a query is entered. Ties are ordered alphabetically. * - name (A–Z, Z–A) - Alphabetical order of the described material's language (``search.content-locale``) — with the default ``cs-CZ`` the Czech alphabet (``č`` after ``c``, ``ch`` after ``h``); records without a name come last. The default when browsing without a query. Every visitor sees the same order, whatever language their browser asks for: the order is a property of the catalogue, not of the reader. * - dating (ascending, descending) - By the earliest (ascending) or latest (descending) dating of the record; records without any dating always come last. Paging is stable under every ordering — a record never moves between pages while the user browses an unchanged index. How ranking works ================= A match is weighted by *where* it occurs. The default tiers, from strongest to weakest: 1. the record's name equals the query (diacritics-exact above diacritics-insensitive), 2. the name begins with the query, 3. the name contains the query as a phrase, 4. the name contains all query words, 5. a **variant name form** matches the same way (the other names of an access point — Praha/Prague; item types marked ``nameVariant: true`` in ``types.yaml``) — the whole ladder repeats one level lower, a preferred name counting roughly five times a variant, 6. labels of referenced records (originators, places, …), 7. the description, 8. any other indexed item. Within a tier, standard fulltext scoring applies — shorter fields matching more of the query score higher, so a record whose name *is* the query outranks a record that merely mentions it. Which facets a section offers ============================= Each facet in ``searchConfig.yaml`` carries an optional ``when`` condition saying where it is offered. Only two forms exist, and **anything else stops the startup** rather than being quietly ignored — a condition the server cannot read used to mean "offered everywhere", which is invisible until a reader is shown a facet belonging to another kind of record. .. code-block:: yaml facets: # bound to one section - when: apuType: FUND type: FULLTEXT source: TITLE # the section, plus a dependency on what another facet has selected - when: all: - apuType: ARCH_DESC - filter: REGISTRY_TYPE value: rejstřík zeměpisný type: MULTI_REF source: REG_GEO_REF ``apuType`` must be one of ``ARCH_DESC``, ``COLLECTION``, ``ENTITY``, ``FINDING_AID``, ``FUND``, ``INSTITUTION``. A facet with no ``when`` at all — or an ``all`` naming no ``apuType`` — belongs to every section. A ``filter``/``value`` condition makes the facet **wait for a selection**: the example above is offered once the reader has chosen that kind of register, so an archival description does not present all seven register facets at once. Several such conditions must all hold; the named facet may have other values selected as well, so long as one of them is the one named. The value may be written as the option's own value or as the label it is displayed under. If the reader then removes the selection the facet was waiting for, the constraint set in that facet is **dropped** — it depended on a choice that is no longer made, and leaving it applied would narrow the results for a reason nothing on the page explains. A home-page tile filtering such a facet must therefore select the value itself, or the startup fails: the tile would otherwise arrive at a search that discards its filter. Ordering a facet's values ========================= ``orderBy`` sorts an enumerated facet's values by frequency (``FREQ``, the default) or alphabetically (``ASC``) — those two and nothing else, or the startup fails: any other word would silently mean ``FREQ``, the opposite of the alphabetical order somebody meant to ask for. ``order`` overrides it for the values an archive wants offered first, whatever their frequency: .. code-block:: yaml - when: apuType: ARCH_DESC type: ENUM source: UNIT_TYPE orderBy: FREQ order: - matrika - kroniky The listed values lead, in the order given; everything else follows in ``orderBy`` order, so the list names a leading run rather than the whole facet. A listed value the data does not contain simply does not appear. An entry may name the option by its value or by the label it is displayed under, so a facet whose values are references can be ordered by the names an archivist thinks in. Explaining a facet's options ============================ Where an option's value does not say what it covers, ``tooltips`` says it. The text is shown when the reader hovers or moves the keyboard to the option, and is read out as the option's description: .. code-block:: yaml - when: apuType: ARCH_DESC type: ENUM source: UNIT_TYPE tooltips: - value: technický výkres tooltip: technické výkresy staveb a výrobků Like ``order``, an entry names the option by its value or by its displayed label. Two entries for one option are not an error — the first is used — but a warning is logged at startup, since the second text would never be seen. Translations go in the sibling ``searchConfig_localization.yaml`` under an ``options`` mapping keyed by the same value — an option has no code of its own, so its value is the key: .. code-block:: yaml facets: en: UNIT_TYPE: options: technický výkres: technical drawings of buildings and products Tuning the weights ================== Weights are deployment configuration — the ``relevance:`` section of ``searchConfig.yaml``. Every key is optional; without the section the built-in defaults above apply. Changes take effect after a restart, **no reindex needed**. .. code-block:: yaml relevance: # Minimum share of query words a record must match (default: all). minimumShouldMatch: 100% # Retry with "any word" when nothing matches (default: true). relaxOnNoHits: true # Minimum fragment length for automatic partial (substring) matching. partialMinLength: 3 # Inflection-aware matching where the content locale has a stemmer # (default: on). Query-side only - toggling needs no reindex. stemming: true # Built-in fields with their default weights. name: { exact: 1000, exactFolded: 800, prefix: 200, phrase: 100, terms: 50, stemmed: 40, wordPrefix: 30, contains: 15 } nameVariants: { exact: 200, exactFolded: 160, prefix: 40, phrase: 20, terms: 10, stemmed: 8, wordPrefix: 8, contains: 4 } refLabels: { phrase: 12, terms: 10 } description: { phrase: 8, terms: 2 } allText: { terms: 1, stemmed: 1 } # Item types promoted above the general-fulltext baseline. items: - source: TITLE phrase: 60 terms: 30 Unknown item-type codes and non-positive weights are reported at startup. To inspect ranking on real data, enable trace logging for the search layer — the server then logs the resolved weight table, the planned query and the scores of the top hits. Limits ====== Protective limits, configurable in ``application.yml``: .. list-table:: :widths: 40 15 45 :header-rows: 1 * - Key - Default - Effect * - ``search.max-window`` - 10 000 - Maximum page window (``from + size``); deeper paging is rejected. * - ``search.track-total-hits-up-to`` - 10 000 - Result totals are exact up to this count; above it the portal shows "more than N results". Clients may request a higher accuracy per query (``totalUpTo``). * - ``search.track-total-hits-max`` - 100 000 - Upper bound for the per-query ``totalUpTo`` override. * - ``search.max-facet-buckets`` - 200 - Hard cap on facet bucket counts. * - ``search.request-timeout`` - 5 s - Per-query time budget. At most 32 query words are used; extra words are ignored.