Search
Workarea applications persist model data to MongoDB, using it as the database of record. However, to provide near-real-time search, Workarea persists many of the same models to Elasticsearch, after first transforming them into a format suitable for search.
Workarea callbacks workers enqueue in response to changes to Mongoid documents, and when run, these workers synchronize these data changes to Elasticsearch.
Workarea also provides query classes which encapsulate the complexity of the various search requests to Elasticsearch needed by Workarea applications. Each search query provides the search results for a given set of query parameters. Results are Mongoid models, initialized directly from a serialized copy in Elasticsearch, so it is not necessary to query MongoDB for results.
Client & Server(s)
Workarea uses a Ruby client to communicate with an Elasticsearch cluster. In Workarea Cloud environments, the Elasticsearch cluster is already provisioned. You must provision your own cluster in other environments.
The client instance, accessed as Workarea.elasticsearch
, provides a Ruby implementation of the Elasticsearch REST APIs.
Workarea.elasticsearch.class
# => Elasticsearch::Transport::Client
In normal operation, you will not use the client directly (favoring higher level APIs), however, direct access to the client is useful for debugging and other secondary use cases. The example below uses the client to print a simple status report for the entire cluster.
puts Workarea.elasticsearch.cat.health(v: true, h: %w(cluster status))
# cluster status
# elasticsearch yellow
Documents, Indexes, Types & Mappings
Types
Elasticsearch uses types to categorize documents according to their fields, or mappings. Workarea uses four types out of the box:
- admin
- help
- storefront
- category
Workarea uses documents of type category to index category queries for use with the percolate query, which can find the matching categories for a given product document. Because category documents are indexed and queried differently than the others, I do not cover them in this guide.
Indexes
Documents of type admin, help, and storefront are stored in separate indexes. Workarea applications use a varying number of Elasticsearch indexes. An index exists for each combination of site name, Rails environment, locale, and Elasticsearch document type.
( Documents of type category are stored in the same indexes as documents of type storefront. )
For example, the indexes for a simple development application could be as follows.
- board_games_direct_development_en_admin
- board_games_direct_development_en_help
- board_games_direct_development_en_storefront
Meanwhile, the following list of indexes could be used in an application with multiple site names (requires the Multi Site plugin), environments, and locales.
- board_games_direct_development_en_admin
- board_games_direct_development_en_help
- board_games_direct_development_en_storefront
- board_games_direct_development_es_admin
- board_games_direct_development_es_help
- board_games_direct_development_es_storefront
- board_games_direct_production_en_admin
- board_games_direct_production_en_help
- board_games_direct_production_en_storefront
- board_games_direct_production_es_admin
- board_games_direct_production_es_help
- board_games_direct_production_es_storefront
- board_games_direct_qa_en_admin
- board_games_direct_qa_en_help
- board_games_direct_qa_en_storefront
- board_games_direct_qa_es_admin
- board_games_direct_qa_es_help
- board_games_direct_qa_es_storefront
- board_games_direct_staging_en_admin
- board_games_direct_staging_en_help
- board_games_direct_staging_en_storefront
- board_games_direct_staging_es_admin
- board_games_direct_staging_es_help
- board_games_direct_staging_es_storefront
- party_games_direct_development_en_admin
- party_games_direct_development_en_help
- party_games_direct_development_en_storefront
- party_games_direct_development_es_admin
- party_games_direct_development_es_help
- party_games_direct_development_es_storefront
- party_games_direct_production_en_admin
- party_games_direct_production_en_help
- party_games_direct_production_en_storefront
- party_games_direct_production_es_admin
- party_games_direct_production_es_help
- party_games_direct_production_es_storefront
- party_games_direct_qa_en_admin
- party_games_direct_qa_en_help
- party_games_direct_qa_en_storefront
- party_games_direct_qa_es_admin
- party_games_direct_qa_es_help
- party_games_direct_qa_es_storefront
- party_games_direct_staging_en_admin
- party_games_direct_staging_en_help
- party_games_direct_staging_en_storefront
- party_games_direct_staging_es_admin
- party_games_direct_staging_es_help
- party_games_direct_staging_es_storefront
Mappings
Elasticsearch mappings are typically declared for an index when the index is created, however, the mapping may be extended at index time, such as when an index's mapping includes dynamic templates.
When creating a new index, Workarea looks for a configuration value declaring the mapping for that index. The configuration keys are named after the different document types. Each configured mapping includes properties and dynamic templates.
( The storefront key declares mappings for the storefront and category types, since both document types are stored in the same indexes. )
For example, the default configuration of the mappings for storefront indexes are shown below.
config.elasticsearch_mappings.storefront = {
category: { properties: { query: { type: 'percolator' } } },
storefront: {
dynamic_templates: [
{
facets: {
path_match: 'facets.*',
mapping: { type: 'keyword' }
}
},
{
numeric: {
path_match: 'numeric.*',
mapping: { type: 'float' }
}
},
{
keywords: {
path_match: 'keywords.*',
mapping: { type: 'keyword' }
}
},
{
sorts: {
path_match: 'sorts.*',
mapping: { type: 'float' }
}
},
{
content: {
path_match: 'content.*',
mapping: { type: 'text', analyzer: 'text_analyzer' }
}
},
{
cache: {
path_match: 'cache.*',
mapping: { index: false }
}
}
],
properties: {
id: { type: 'keyword' },
type: { type: 'keyword' },
slug: { type: 'keyword' },
suggestion_content: { type: 'string', analyzer: 'text_analyzer' }
}
}
}
Because the storefront mappings depend heavily on dynamic templates, the method Workarea::Search::Storefront.ensure_dynamic_mappings
is available, which creates a product document, indexes it, and then removes it. This process helps reduce errors in a new environment where the index-time mappings have not been created yet.
Document Interface
The Workarea::Elasticsearch::Document
module provides a Ruby interface to represent behavior shared by Elasticsearch documents of all types. Each class that includes this module represents a specific type. Calling Document.all
returns a list of the more specific classes.
puts Workarea::Elasticsearch::Document.all
# Workarea::Search::Admin
# Workarea::Search::Help
# Workarea::Search::Storefront
The classes listed above are used almost exclusively for type and index level concerns (see below), while the descendants of these classes (covered under Search Models) are used primarily for document level concerns.
( Workarea::Search::Help
has no descendants and is used directly to save and destroy documents. )
These classes respond to .type
and .mappings
, which describe the type of documents for which they are responsible. (Note that .mappings
returns the Workarea configuration, not the actual mappings on the index.)
Workarea::Elasticsearch::Document.all.map(&:type)
# => [:admin, :help, :storefront]
pp Workarea::Search::Help.mappings
# {:help=>
# {:dynamic_templates=>
# [{:facet_values=>
# {:path_match=>"facets.*",
# :mapping=>{:type=>"string", :analyzer=>"keyword"}}}],
# :properties=>
# {:id=>{:type=>"string", :index=>"not_analyzed"},
# :name=>{:type=>"string", :analyzer=>"text_analyzer"},
# :body=>{:type=>"string", :analyzer=>"text_analyzer"},
# :created_at=>{:type=>"date"}}}}
The following methods are used to create, delete, and reset all indexes for the particular document type. Remember that each document type may be stored on many indexes depending on the number of sites, environments, and locales.
.create_indexes!
.delete_indexes!
.reset_indexes!
Every Elasticsearch document class has a current index. The example below demonstrates the current index is determined by the combination of current site name, Rails environment, locale, and document type.
Workarea::Search::Storefront.current_index.name
# => "board_games_direct_development_en_storefront"
Workarea::Search::Storefront.current_index.url
# => "http://localhost:9200/board_games_direct_development_en_storefront"
Workarea.config.site_name
# => "Board Games Direct"
Rails.env
# => "development"
I18n.locale
# => :en
Workarea::Search::Storefront.type
# => :storefront
The current index is an instance of Workarea::Elasticsearch::Index
, another abstraction provided by Workarea.
Workarea::Search::Storefront.current_index.class
# => Workarea::Elasticsearch::Index
I do not cover this abstraction in detail because it is used internally and rarely invoked by application code. A Search model initializes instances of Index
when needed and delegates index operations to its current_index
.
Indexing
Most search indexing is performed by callbacks workers running in response to changes to application documents, and by workers run on a schedule. Internally, workers use search models to transform the affected application documents into search documents, which involves serializing in-memory objects into strings. The search models then index the transformed documents into Elasticsearch.
To a lesser extent, rake tasks and seeds are also used to index documents for search.
Workers
Various workers are used to index documents into Elasticsearch. Many of these are callbacks workers that run in response to life cycle changes on application documents (Mongoid documents) and use search models to forward Mongoid changes to Elasticsearch.
For example, review the implementation of Workarea::IndexPage
, below.
module Workarea
class IndexPage
include Sidekiq::Worker
include Sidekiq::CallbacksWorker
sidekiq_options(
enqueue_on: { Content::Page => [:save, :destroy] },
unique: :until_executing
)
def perform(id)
page = Content::Page.find(id)
Search::Storefront::Page.new(page).save
rescue Mongoid::Errors::DocumentNotFound
Search::Storefront::Page.new(
Content::Page.new(id: id)
).destroy
end
end
end
when a Workarea::Content::Page
is saved or destroyed, Workarea::IndexPage
is enqueued with the id of the saved or destroyed content page. When this worker runs, it looks up the affected content page and creates an instance of Search::Storefront::Page
from it. Then the worker uses Search::Storefront::Page#save
or Search::Storefront::Page#destroy
to create and index, or delete, the corresponding search document.
The Workarea::IndexAdminSearch
worker follows the same pattern, but it is enqueued on save, touch, or destroy of any application document. The worker uses the Search::Admin.for
factory to initialize the correct search model from the affected application document.
The Admin UI inlines IndexAdminSearch
for the duration of each Admin request so that model changes applied through the Admin UI are applied inline rather than asynchronously.
module Workarea
module Admin
class ApplicationController < Workarea::ApplicationController
# ...
around_action :inline_search_indexing
# ...
private
# ...
def inline_search_indexing
Sidekiq::Callbacks.inline(IndexAdminSearch) { yield }
end
# ...
end
end
end
Some callbacks workers delegate to other workers rather than using a search model directly. The workers that are delegated to implement a .perform
class method, which typically receives the affected Mongoid model instance (rather than an id) as its argument.
For example, changes to catalog variants and product images cause the parent products to be re-indexed into the Storefront indexes. To do so, the Workarea::IndexProductChildren
worker determines which parent product is affected and uses IndexProduct.perform
to re-index the product.
module Workarea
class IndexProductChildren
include Sidekiq::Worker
include Sidekiq::CallbacksWorker
sidekiq_options(
enqueue_on: {
Catalog::Variant => [:save, :destroy],
Catalog::ProductImage => [:save, :destroy],
with: -> { [_parent.id.to_s] }
},
unique: :until_executing
)
def perform(id)
product = Catalog::Product.find(id) rescue nil
IndexProduct.perform(product) if product.present?
end
end
end
Similarly, changes to catalog categories cause affected products to be re-indexed. The Workarea::IndexCategoryChanges
determines which products are affected and uses BulkIndexProducts.perform
to re-index them in a single request.
module Workarea
class IndexCategoryChanges
include Sidekiq::Worker
include Sidekiq::CallbacksWorker
sidekiq_options(
enqueue_on: { Catalog::Category => :save, with: -> { [changes] } },
ignore_if: -> { changes['product_ids'].blank? },
unique: :until_executing
)
def perform(changes)
if changes['product_ids'].present?
previous_ids = changes['product_ids'].first || []
new_ids = changes['product_ids'].second || []
require_index_ids = (previous_ids - new_ids) + (new_ids - previous_ids)
BulkIndexProducts.perform(require_index_ids)
end
end
end
end
Other indexing workers run on a schedule instead of in response to model changes. Examples are Workarea::CleanOrders
and Workarea::KeepProductIndexFresh
.
Finally, some workers are run neither as callbacks nor on a schedule. These workers must be run manually, usually from another worker. Examples are Workarea::BulkIndexProducts
and Workarea::BulkIndexSearches
.
Search Models
Search models are initialized from Mongoid documents and can save and destroy corresponding search documents to and from relevant Elasticsearch indexes. Search documents are responsible for transforming Mongoid documents into search documents, and in the process of doing so, they serialize the entire Mongoid document for storage into Elasticsearch.
Initializing
Workarea::Search::Admin
is an abstract superclass whose subclasses are search models that index documents of type admin.
puts Workarea::Search::Admin.descendants
# Workarea::Search::Admin::CatalogCategory
# Workarea::Search::Admin::CatalogProduct
# Workarea::Search::Admin::Content
# Workarea::Search::Admin::ContentAsset
# Workarea::Search::Admin::ContentPage
# Workarea::Search::Admin::InventorySku
# Workarea::Search::Admin::Navigation
# Workarea::Search::Admin::NavigationMenu
# Workarea::Search::Admin::Order
# Workarea::Search::Admin::PaymentTransaction
# Workarea::Search::Admin::PricingDiscount
# Workarea::Search::Admin::PricingSku
# Workarea::Search::Admin::Release
# Workarea::Search::Admin::User
The factory method Workarea::Search::Admin.for
will create an instance of one of the above search models for a given Mongoid model (see example in workers section, above).
Similarly, Workarea::Search::Storefront
is an abstract superclass whose subclasses are search models that index documents of type storefront.
puts Workarea::Search::Storefront.descendants
# Workarea::Search::Storefront::Category
# Workarea::Search::Storefront::Page
# Workarea::Search::Storefront::Product
# Workarea::Search::Storefront::Search
Workarea::Search::Help
has no descendants and is used as the search model for documents of type help.
Workarea::Search::Help.descendants
# => []
Initialize a search model with an application document (a Mongoid model).
product = Workarea::Catalog::Product.create!(name: 'Escape the Room')
product_admin_search_model = Workarea::Search::Admin::CatalogProduct.new(product)
# or use the factory for 'admin' documents
product_admin_search_model = Workarea::Search::Admin.for(product)
product_admin_search_model.class
# => Workarea::Search::Admin::CatalogProduct
Search models return a type
, which describes the Mongoid document type rather than the Elasticsearch document type. In the case of Admin and Storefront search models, many different Mongoid types map to the same Elasticsearch types.
# Mongoid "type"
product_admin_search_model.type
# => "product"
# Elasticsearch "type"
product_admin_search_model.class.type
# => :admin
The following examples use the same Mongoid model from above to create and explore a Storefront search model.
product_storefront_search_model = Workarea::Search::Storefront::Product.new(product)
product_storefront_search_model.class
# => Workarea::Search::Storefront::Product
product_storefront_search_model.type
# => "product"
product_storefront_search_model.class.type
# => :storefront
A search model provides access to the original Mongoid model.
product_storefront_search_model.model.class
# => Workarea::Catalog::Product
product_storefront_search_model.model.name
# => "Escape the Room"
product_storefront_search_model.model.id
# => "C204935012"
Be aware that a search model and its originating Mongoid model have different, albeit similar, IDs.
product_storefront_search_model.id
# => "product-C204935012"
product_storefront_search_model.catalog_id
# => "C204935012"
product_storefront_search_model.model.id
# => "C204935012"
Since Workarea 3.5, some Storefront search documents are specific to a release. In these cases, the search model ID also contains the release ID:
# create a release
release = Workarea::Release.create!(name: 'Catalog Cleanup')
release.id.to_s
# => "5d8a8ec63e474d3333402efb"
# change the product within that release
Workarea::Release.with_current(release) do
Workarea::Catalog::Product
.find(product.id)
.update_attribute(:name, 'Escape the Room 2')
end
# create a new, release-specific search model for the product
Workarea::Release.with_current(release) do
product = Workarea::Catalog::Product.find(product.id)
end
product_storefront_search_model = Workarea::Search::Storefront::Product.new(product)
# view the search model ID
product_storefront_search_model.id
# => "product-C204935012-5d8a8ec63e474d3333402efb"
Saving & Destroying
Use the save
method to create and index a search document, and use the destroy
method to delete a search document.
product_storefront_search_model.save
product_storefront_search_model.destroy
These operations may affect multiple documents (in multiple indexes) if the application has multiple locales or releases.
Creating Search Documents
When search models save
documents, they must first create a document that is suitable for Elasticsearch. They do so by transforming the Mongoid model, and in many cases, aggregating multiple related models. For example, a searchable product requires catalog, pricing, and inventory data. Each search model implements as_document
and as_bulk_document
for this purpose.
The following examples create Admin and Storefront search documents from the same Mongoid source document. Notice how the fields of each search document are different.
product_admin_search_model = Workarea::Search::Admin::CatalogProduct.new(product)
product_admin_search_document = product_admin_search_model.as_document
puts JSON.pretty_generate(product_admin_search_document)
# {
# "id": "product-C204935012",
# "name": "Escape the Room 2",
# "facets": {
# "status": "inactive",
# "type": "product",
# "tags": [
#
# ],
# "upcoming_changes": [
# "5d8a8ec63e474d3333402efb"
# ],
# "category": "New",
# "category_id": [
#
# ],
# "on_sale": false,
# "inventory_policies": [
#
# ],
# "issues": [
# "No Images",
# "No Description",
# "No Variants"
# ],
# "template": "generic"
# },
# "created_at": "2019-09-24 21:43:01 UTC",
# "updated_at": "2019-09-24 21:43:01 UTC",
# "keywords": [
# "c204935012"
# ],
# "search_text": [
# "C204935012",
# "Escape the Room 2",
# "product"
# ],
# "jump_to_text": "Escape the Room 2 (C204935012)",
# "jump_to_search_text": [
# "C204935012",
# "Escape the Room 2",
# "product"
# ],
# "jump_to_position": 3,
# "jump_to_route_helper": "catalog_product_path",
# "jump_to_param": "escape-the-room",
# "releasable": true
# }
product_storefront_search_model = Workarea::Search::Storefront::Product.new(product)
product_storefront_search_document = product_storefront_search_model.as_document
puts JSON.pretty_generate(product_storefront_search_document)
# {
# "id": "product-C204935012-5d8a8ec63e474d3333402efb",
# "type": "product",
# "slug": "escape-the-room",
# "active": {
# "now": false
# },
# "release_id": "5d8a8ec63e474d3333402efb",
# "changeset_release_ids": [
# "5d8a8ec63e474d3333402efb"
# ],
# "suggestion_content": "Escape the Room 2 New, Phone Cases, Gaming, Fiction, Puzzles, Board Games ",
# "created_at": "2019-09-24 21:43:01 UTC",
# "updated_at": "2019-09-24 21:43:01 UTC",
# "facets": {
# "category": "New",
# "category_id": [
#
# ],
# "on_sale": false,
# "inventory_policies": [
#
# ]
# },
# "numeric": {
# "price": [
# 0.0
# ],
# "inventory": 0,
# "variant_count": 0
# },
# "keywords": {
# "catalog_id": "c204935012",
# "sku": [
#
# ],
# "name": "escape the room 2"
# },
# "sorts": {
# "price": 0.0,
# "orders_score": 0,
# "views_score": 0,
# "inventory_score": 1
# },
# "content": {
# "name": "Escape the Room 2",
# "category_names": "New, Phone Cases, Gaming, Fiction, Puzzles, Board Games",
# "description": "",
# "details": " ",
# "facets": ""
# },
# "cache": {
# "image": "/product_images/placeholder/small_thumb.jpg?c=1567110494",
# "pricing": [
#
# ],
# "inventory": [
#
# ]
# }
# }
The fields of a search document should match the mapping for its type. Some fields appear in the mapping explicitly, as properties, while other match implicitly, via dynamic templates.
Some fields are used only for storage, not search. For example, the cache.image, cache.pricing, and cache.inventory fields above are all configured to be stored but not indexed.
config.elasticsearch_mappings.storefront = {
# ...
storefront: {
dynamic_templates: [
# ...
{
cache: {
path_match: 'cache.*',
mapping: { index: false }
}
}
],
properties: {
# ...
}
}
}
Cached fields are used when loading results to avoid additional queries to MongoDB.
Much of each search model's implementation is made up of methods used to compose the hash returned from as_document.
Transformation "Helpers"
Workarea provides several classes to help construct the as_document
hash.
Workarea::Search::Admin::Releasable
is included in search models for releasable models. It includes additional fields in the as_document
hash related to releases. The following example lists classes that include this module.
releasables = Workarea::Search::Admin.descendants.select do |klass|
klass.included_modules.include?(Workarea::Search::Admin::Releasable)
end
puts releasables.map(&:to_s).sort
# Workarea::Search::Admin::CatalogCategory
# Workarea::Search::Admin::CatalogProduct
# Workarea::Search::Admin::Content
# Workarea::Search::Admin::ContentPage
# Workarea::Search::Admin::NavigationMenu
# Workarea::Search::Admin::PricingDiscount
# Workarea::Search::Admin::PricingSku
Several other POROs exist to help construct specific values for the as_document hash.
Workarea::Search::FacetValues
Workarea::Search::HashText
Workarea::Search::OrderText
Workarea::Search::UserText
Serialization & De-Serialization
Workarea provides Workarea::Elasticsearch::Serializer
for serializing in-memory models into strings suitable for storage in Elasticsearch, and for de-serializing those strings back into models without needing to query MongoDB.
product = Workarea::Catalog::Product.first
product.class
# => Workarea::Catalog::Product
product.id
# => "006CBBCD90"
product.name
# => "Fantastic Linen Pants"
Serialization passes the object through Mongoid::Document.as_document
(if a Mongoid document), then Marshal.dump
, and finally Base64.encode64
.
serialized_product = Workarea::Elasticsearch::Serializer.serialize(product)
serialized_product.class
# => Hash
serialized_product['model_class']
# => "Workarea::Catalog::Product"
serialized_product['model']
# => "BAhDOhNCU09OOjpEb2N1bWVudHsVSSIIX2lkBjoGRVRJIg8wMDZDQkJDRDkw...
De-serialization reverses the process, passing the string through Base64.decode64
, then Marshal.load
, and finally Mongoid::Factory.from_db
to re-create the Mongoid document instance.
deserialized_product = Workarea::Elasticsearch::Serializer.deserialize(serialized_product)
deserialized_product.class
# => Workarea::Catalog::Product
deserialized_product.id
# => "006CBBCD90"
deserialized_product.name
# => "Fantastic Linen Pants"
Rake Tasks
Workarea provides several Rake tasks to manually re-index all search indexes or indexes for particular document types. The tasks are defined in workarea-core/lib/tasks/search.rake
.
You can run the tasks as commands.
$ bin/rails workarea:search_index:all
$ bin/rails workarea:search_index:admin
$ bin/rails workarea:search_index:storefront
$ bin/rails workarea:search_index:help
Or invoke the tasks programatically.
Rake::Task['workarea:search_index:all'].invoke
Rake::Task['workarea:search_index:admin'].invoke
Rake::Task['workarea:search_index:storefront'].invoke
Rake::Task['workarea:search_index:help'].invoke
Regardless of how you execute them, these tasks each inline all Sidekiq workers for the duration of the task. As of Workarea 3.3, you can prevent this behavior by setting the environment variable INLINE
to false
. For example:
$ INLINE=false bin/rails workarea:search_index:all
Seeds
Running seeds drops all MongoDB and Elasticsearch data for the current environment and primes both databases with data. Running seeds therefore resets all Elasticsearch indexes.
You can run seeds as a command.
$ bin/rails db:seed
Or programatically.
Workarea::Seeds.run
Search Queries
Workarea provides a variety of query classes which are responsible for complicated reads. This includes Elasticsearch searches, for which Workarea provides a variety of search queries.
Search queries are initialized with params and construct and perform an Elasticsearch request body search. The query provides access to the raw Elasticsearch response in addition to "loaded" results, which returns the results as Mongoid documents, initialized from the serialized model cache within each Elasticsearch document.
Each search query instance therefore represents the Elasticsearch request and response for a given set of params. UI code often wraps a query instance in a view model and presents the results.
Types & Initialization
Search queries are classes that include Workarea::Search::Query
. The following example introspects the Ruby object space for search query classes.
searches = ObjectSpace.each_object(Class).select do |klass|
klass.included_modules.include?(Workarea::Search::Query)
end
puts searches.map(&:to_s).sort
# Workarea::Search::AdminAssets
# Workarea::Search::AdminCategories
# Workarea::Search::AdminDiscounts
# Workarea::Search::AdminInventorySkus
# Workarea::Search::AdminOrders
# Workarea::Search::AdminPages
# Workarea::Search::AdminPaymentTransactions
# Workarea::Search::AdminPricingSkus
# Workarea::Search::AdminProducts
# Workarea::Search::AdminReleasables
# Workarea::Search::AdminSearch
# Workarea::Search::AdminUsers
# Workarea::Search::Categorization
# Workarea::Search::CategoryBrowse
# Workarea::Search::HelpSearch
# Workarea::Search::ProductSearch
# Workarea::Search::RelatedHelp
# Workarea::Search::RelatedProducts
Each search query searches for results of a particular Elasticsearch document type. The following example groups the search queries by type.
pp searches.group_by(&:document)
# {Workarea::Search::Storefront=>
# [Workarea::Search::RelatedProducts,
# Workarea::Search::ProductSearch,
# Workarea::Search::CategoryBrowse,
# Workarea::Search::Categorization],
# Workarea::Search::Help=>
# [Workarea::Search::RelatedHelp,
# Workarea::Search::HelpSearch],
# Workarea::Search::Admin=>
# [Workarea::Search::AdminUsers,
# Workarea::Search::AdminSearch,
# Workarea::Search::AdminReleasables,
# Workarea::Search::AdminProducts,
# Workarea::Search::AdminPricingSkus,
# Workarea::Search::AdminPaymentTransactions,
# Workarea::Search::AdminPages,
# Workarea::Search::AdminOrders,
# Workarea::Search::AdminInventorySkus,
# Workarea::Search::AdminDiscounts,
# Workarea::Search::AdminCategories,
# Workarea::Search::AdminAssets]}
Each search query instance is initialized with params.
product_admin_search_query = Workarea::Search::AdminProducts.new(q: 'escape')
product_admin_search_query.class
# => Workarea::Search::AdminProducts
product_admin_search_query.class.document
# => Workarea::Search::Admin
product_admin_search_query.class.document.type
# => :admin
Results
The #total
and #stats
methods return meta data about the results.
product_admin_search_query.total
# => 1
pp product_admin_search_query.stats
# {"upcoming_changes"=>
# {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=>[]},
# "template"=>
# {"doc_count_error_upper_bound"=>0,
# "sum_other_doc_count"=>0,
# "buckets"=>[{"key"=>"generic", "doc_count"=>1}]},
# "color"=>
# {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=>[]},
# "size"=>
# {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=>[]},
# "price"=>
# {"buckets"=>
# [{"key"=>"*-9.99", "to"=>9.99, "doc_count"=>0},
# {"key"=>"10.0-19.99", "from"=>10.0, "to"=>19.99, "doc_count"=>0},
# {"key"=>"20.0-29.99", "from"=>20.0, "to"=>29.99, "doc_count"=>0},
# {"key"=>"30.0-39.99", "from"=>30.0, "to"=>39.99, "doc_count"=>0},
# {"key"=>"40.0-49.99", "from"=>40.0, "to"=>49.99, "doc_count"=>0},
# {"key"=>"50.0-59.99", "from"=>50.0, "to"=>59.99, "doc_count"=>0},
# {"key"=>"60.0-69.99", "from"=>60.0, "to"=>69.99, "doc_count"=>0},
# {"key"=>"70.0-79.99", "from"=>70.0, "to"=>79.99, "doc_count"=>0},
# {"key"=>"80.0-89.99", "from"=>80.0, "to"=>89.99, "doc_count"=>0},
# {"key"=>"90.0-99.99", "from"=>90.0, "to"=>99.99, "doc_count"=>0},
# {"key"=>"100.0-*", "from"=>100.0, "doc_count"=>0}]},
# "type"=>
# {"doc_count_error_upper_bound"=>0,
# "sum_other_doc_count"=>0,
# "buckets"=>[{"key"=>"product", "doc_count"=>1}]},
# "issues"=>
# {"doc_count_error_upper_bound"=>0,
# "sum_other_doc_count"=>0,
# "buckets"=>
# [{"key"=>"No Description", "doc_count"=>1},
# {"key"=>"No Images", "doc_count"=>1},
# {"key"=>"No Variants", "doc_count"=>1}]},
# "status"=>
# {"doc_count_error_upper_bound"=>0,
# "sum_other_doc_count"=>0,
# "buckets"=>[{"key"=>"inactive", "doc_count"=>1}]},
# "tags"=>
# {"doc_count_error_upper_bound"=>0, "sum_other_doc_count"=>0, "buckets"=>[]}}
The #response
method returns the raw response. Looking at the response, you can see the serialized model data stored within the Elasticsearch document source.
pp product_admin_search_query.response
# {"took"=>23,
# "timed_out"=>false,
# "_shards"=>{"total"=>5, "successful"=>5, "failed"=>0},
# "hits"=>
# {"total"=>1,
# "max_score"=>nil,
# "hits"=>
# [{"_index"=>"try_search_development_en_admin",
# "_type"=>"admin",
# "_id"=>"product-273E095F5A",
# "_score"=>6.044283,
# "_source"=>
# {"id"=>"product-273E095F5A",
#
# # ...
#
# "model_class"=>"Workarea::Catalog::Product",
# "model"=>
# "BAhDOhNCU09OOjpEb2N1bWVudHsSSSIIX2lkBjoGRVRJIg8yNzNFMDk1RjVB\n" +
# "BzsGVDobQHVuY29udmVydGFibGVfdG9fYnNvblRJIgl0YWdzBjsGVFsASSIL\n" +
# "YWN0aXZlBjsGVFRJIhhzdWJzY3JpYmVkX3VzZXJfaWRzBjsGVFsASSIMZGV0\n" +
# "YWlscwY7BlRDOwB7BkkiB2VuBjsGVEM7AHsASSIMZmlsdGVycwY7BlRDOwB7\n" +
# "BkkiB2VuBjsGVEM7AHsASSINdGVtcGxhdGUGOwZUSSIMZ2VuZXJpYwY7BlRJ\n" +
# "IhBwdXJjaGFzYWJsZQY7BlRUSSIJbmFtZQY7BlRDOwB7BkkiB2VuBjsGVEki\n" +
# "FEVzY2FwZSB0aGUgUm9vbQY7BlRJIgxkaWdpdGFsBjsGVEZJIglzbHVnBjsG\n" +
# "VEkiFGVzY2FwZS10aGUtcm9vbQY7BlRJIg91cGRhdGVkX2F0BjsGVEl1OglU\n" +
# "aW1lDfJeHcBolXe5BjoJem9uZUkiCFVUQwY7BkZJIg9jcmVhdGVkX2F0BjsG\n" +
# "VEl1OwgN8l4dwGiVd7kGOwlJIghVVEMGOwZG\n"},
# "sort"=>[6.044283, 1503513983497]}]},
# "aggregations"=> # ...}
The #results
method returns the "loaded" results, which de-serializes each result into a Mogoid model. These results are suitable for display in the UI.
product_admin_search_query.results.class
# => Workarea::PagedArray
product_admin_search_query.results.to_a.map(&:class)
# => [Workarea::Catalog::Product]
product_admin_search_query.results.first.id
# => "273E095F5A"
product_admin_search_query.results.first.name
# => "Escape the Room"
Product Results
Search queries that return products, such as ProductSearch
, CategoryBrowse
, and RelatedProducts
include the LoadProductResults
module, which changes the behavior of results
.
For each result, these queries return a hash of objects rather than a single object.
product_search = Workarea::Search::ProductSearch.new(q: 'marble')
product_search.results.class
# => Workarea::PagedArray
product_search.results.to_a.map(&:class)
# => [Hash, Hash, Hash, Hash, Hash, Hash, Hash, Hash, Hash]
The following examples show the keys in this hash and the type of each value.
product_search.results.first.keys
# => [:id, :catalog_id, :model, :option, :pricing, :inventory]
pp Hash[product_search.results.first.map { |k,v| [k, v.class] }]
# {:id=>String,
# :catalog_id=>String,
# :model=>Workarea::Catalog::Product,
# :option=>NilClass,
# :pricing=>Workarea::Pricing::Collection,
# :inventory=>Workarea::Inventory::Collection}
product_search.results.first[:catalog_id]
# => "29C9ECAAF2"
product_search.results.first[:model].name
# => "Practical Marble Clock"
Composing the Search Request Body
As mentioned above, search queries perform an Elasticsearch request body search. These search requests use a request body constructed from the Elasticsearch query DSL.
Search queries provide a Ruby interface for composing these request bodies. The body
method is responsible for returning a hash which represents the request body. Many other methods are potentially used to compose this final hash. Below is the default implementation of Workarea::Search::Query#body
.
module Workarea
module Search
module Query
# ...
def body
{
query: query,
post_filter: post_filter,
aggs: aggregations
}
.merge(additional_options)
.delete_if { |_, v| v.blank? }
end
# ...
end
end
end
Each search query extends or implements the methods necessary to produce the desired request body. As you can see above, the methods query
, post_filter
, and aggregations
contribute directly to the body. The additional_options
method, calls a method for each search query option included in Workarea.config.search_query_options
, allowing for configuration of the query builder.
Workarea.config.search_query_options
# => ["sort", "size", "from", "suggest"]
Many search queries include some of the following modules, which help to build up the desired request body.
Workarea::Search::CategorizationFiltering
Workarea::Search::Facets
Workarea::Search::Pagination
Workarea::Search::LoadProductResults
Workarea::Search::ProductDisplayRules
Workarea::Search::QuerySuggestions
Workarea::Search::ProductRules
Workarea::Search::AdminIndexSearch
Workarea::Search::AdminSorting
Help Us Improve this Doc
Was this helpful? Open a GitHub issue to report a problem with this doc, suggest an improvement, or otherwise provide feedback. Thanks!