Wednesday 2 August 2017

Steps to implement a search in S&P With AEM

Set up
  • Hosting: Hosting the search tool.
  • Indexing: Indexing the data.
  • Index frequency : Configuring the index.       

Configure
  • Metadata - Used for faceting, sorting, ranking, relevance.
  • Breadcrumbs - for a better navigation.
  • Pagination - for a better navigation.
  • Recent Searches - Search assist.
  • Search suggestions(Did you mean) - Search assist.
  • Auto complete - Search assist.

Fine-tune
  • Promotion: Normal, Based on search result, we can promote.
  • Dictionaries : Configure for better results.
  • Banners : any organizational promotion.
  • Redirects : Send user to a page.

Read more here

Related 
Steps to implement any search technology
AEM Dispatcher, why it  is needed?
AEM Desktop App
Figure out the best search technology or tool
Steps to implement search in Solr
Quality of Search - fine tuning search implementation
 
 
 

How do we select the best search tool for the site?

There are many search engines, some are open source and some are paid. The trend in search technology shows an affinity towards cognitive and artificial intelligence now.

  • Solr
  • Elasticsearch
  • Google Search Appliance (GSA)
  • Oracle Endeca
  • Microsoft FAST
  • Attivio
  • Sinequa
  • Coveo
  • IBM Watson
  • Amazon CloudSearch
  • SharePoint Search
  • HP Autonomy

The common Big data used with searches are:
  • Apache Hadoop
  • Cloudera
  • Hortonworks
Search Trend Evolution
Keyword Search -> Semantic -> Contextual -> Cognitive -> (Human Brain)

Enterprise search now uses natural language processing and machine learning which dramatically improve the results relevancy and completeness.
Here cognitive search is based on Artificial Intelligence.

Which search is the best?

I always suggest the one that meets your needs in your environment and requirements.

Process of identifying the needs

Auditing the current system
Understand the current system and new requirements.
OS & Systems
Think of operating system to be hosted. On-Premise Vs Cloud Vs Hybrid
Dev tools
Think on the tools to develop while search is getting revamped
Repositories
Think about content repository and ensure product upgrade history is clean.
Security
Are there any new security levels to be added?
Content
Think on the content when search system upgrades are in place
Users
External Vs internal? The behavior & type of users.

Understand any data
Search solutions must connect to and ingest data from a wide variety of source. For e.g.: Data types ranging from images, video, audio, and machine data such as from internet-of-things (IoT) devices.

Scale to handle big
Now the data is in peta bytes which resides in distributed architectures.

The migration process
Provide transparent, solid experience to the end users, Educate the users about new system.

Install base and revenue history
A proven stream of revenue generated by customer adoption of its solution and installation market presence.

Cross-domain standalone solution, Employ AI technologies
The solution is self-sufficient, general-purpose cross domain one now. Understand and organize data, predict, improve relevancy, and automatically tune the relevancy of results over time using AI Techniques.

Allow developers to customize search applications
Currently search vendors provide SDKs, Apis, and in some cases visual design tools to customize search to a maximum extend.

List out your semi finalists then finalists. Then rematch the requirement with finalists and decide the winner.

Read More
Steps to implement any search technology
AEM Dispatcher, why it  is needed?
AEM Desktop App
Figure out the best search technology or tool
Steps to implement search in Solr
Quality of Search - fine tuning search implementation

Quality of search - Fine tuning the search

There are many search engines, some are open source and some are paid. The trend in search technology shows an affinity towards cognitive and artificial intelligence now.
  • Solr
  • Elasticsearch
  • Google Search Appliance (GSA)
  • Oracle Endeca
  • Microsoft FAST
  • Attivio
  • Sinequa
  • Coveo
  • IBM Watson
  • Amazon CloudSearch
  • SharePoint Search
  • HP Autonomy
The common Big data used with searches are:
  • Apache Hadoop
  • Cloudera
  • Hortonworks
Quality of search(QoS)
Let us see how we can make a search effective. QoS is a term used with search implementation fine tuning. Once a search is implemented, next steps are to ensure the search performance. There are many ways we can do the fine tuning.

Query cleaning: 
Before search hits the index, ensure the query is cleaned and aligned to your index.

Add analytics:
Ensure you are using analytics hand in hand with search, which will definitely improve the effectiveness of search.

Rules: 
Rules like ranking, relevance etc can be adjusted to fine tune the search results.

Proper metadata mapping:
Re-verify the metadata mapping for index to ensure everything is done perfectly.

Dictionaries, Synonyms:
Some times we miss to configure dictionaries which will restrict the user from synonym search. Words with similar meaning does not come in result set at all. Confirm you have configured them well.

Excluded words:
Ensure the excluded word list is configured so that none of the unwanted queries fetch the result.

Restrictions:
There are cases where we have to skip some pages from search results. Ensure this restrictions are working as expected.

Once all above conditions are met, next step is to verify the search functionality ourselves to confirm the 'recent searches', 'search suggestion', 'auto complete'/ 'type ahead', 'null search results' are functioning as expected.

Finally don't miss to check the reports like Terms report, null search report, search requests, index reports, analytics reports,  crawl reports, content request reports.

Usually search technology is treated as 'once implement then ignore'. My recommendation is keep improving the configurations by constantly watching the reports often.

Read More
Steps to implement any search technology
AEM Dispatcher, why it  is needed?
AEM Desktop App
Figure out the best search technology or tool
Steps to implement search in Solr
Quality of Search - fine tuning search implementation

Monday 10 July 2017

Search promote Rules

Rules:
Rules helps us to validate a condition before/during/after search execution.
Ensure to use meaningful names while creating rules.



Query Cleaning Rules(QCR)
QCR helps to clean up the query triggered from front end and process something specific before the actual query starts execution.

Example 1: Say we need to set a JSON template for transport layer, below query helps for the same.

Name: Set Json Transport Template

If all of the following conditions are met:
    Every query
Perform the following actions:
   Set backend parameter sp_t to value json.tpl

Example 2: Say we have both content and product searches, to enable the content search we can set up a rule as below,

Name: Content Search

   If all of the following conditions are met:
    Query segment equal value content
Perform the following actions:
   Set custom variable content to value 1

So if the query parameter contains term segment='content', the query gets executed and it sets a variable content = '1' in backend, so that in our template, we can redirect the code to retrieve value for content search by testing this variable.

Direct Hits(DH)
This rules gets executed directly on meeting the condition.

Query Pattern:
website_URL: term1, term2, term3
In this case, when the query term contains term1 or term2 or term 3, the website_URL gets executed. Search gets redirected to the url.

Pre-Search Rules(PSR)
This rules are useful when we need to validate something before the search starts.

Example 1
Name: Load Simulator

If all of the following conditions are met:
    In simulation mode
Perform the following actions:
   Set targeted template to simulator

Example 2 
Name: Load RSS Feeds

   If all of the following conditions are met:
    Query q contains value rss
Perform the following actions:
   Set targeted template to feed

Example 3 

Name: Set Default Content Template
   If all of the following conditions are met:
    Custom variable content equal value 1
Perform the following actions:
   Set targeted template to Content_XML
   
Post Search Rules (PSR)
This helps to adjust the result post the search execution(Before displaying on screen). Say we need to remove some unwanted data from search results, in such cases this rule can be used.

Example 1
Name: check if discontinued items exists in results

If all of the following conditions are met:
    Custom variable discontinued not equal value 0
Perform the following actions:
   Set template's backend parameter sp_x_16 in template *targeted for search *primary to value discontinued
   Set template's backend parameter sp_q_16 in template *targeted for search *primary to value 1
   
Business Rules(BR)
Business rules helps to execute some specific business rules on search. Say we need to display a promotonal banner on top of the search results.

Example 1
Name: Business Rule Ferrari

If all of the following conditions are met:
   Keyword equal Ferrari
Perform the following actions:
   Set banner in area top to banner Ferrari - the top one

Example 2 
Name:  Promote Specific Items

   If all of the following conditions are met:
   Every query
Perform the following actions:
   Push group sp_cs = UTF-8 sp_p = all sp_q = * spc_no_field_table = 1 Product-Type = Product for search default to position 1
   Push group sp_cs = UTF-8 sp_p = all sp_q = * spc_no_field_table = 1 Product-Type = Product2 for search default to position 2
   Push group sp_cs = UTF-8 sp_p = all sp_q = * spc_no_field_table = 1 Product-Type = Product3 for search default to position 3


Ranking Rules
Ranking Rules allow you to modify the relative positioning of Search results based on contained meta tag content and/or related Adobe Analytics metrics.

Example 1
(Meta Tag) salesrank 10 - Maximum Importance

Set rank
regexp -[[:digit:]][[:digit:]][[:digit:]][[:digit:]] -1


Thursday 29 June 2017

Adobe Search and Promote Menus Explained



Through this post, we will try to explain the main menus and their functionalities here.

Home:


This menu holds the dashboard for viewing overall search system for a specific account. It includes below sections.

  • Index Information,
  • Terms Report,
  • Null Search Terms Report,
  • Crawl Performance,
  • Search Form,
  • Recent Changes,
  • Alerts


Design:
This menu helps to adjust the design specific search configurations on a page. It includes,

  • Navigation - Facets, Facet Rails(Group of facets), Breadcrumbs, Page Navigation, Menus, Recent Searches
  • Templates - Section where we can create new templates. This templates includes both transport(.tpl) and presentation(.tmpl) templates.
  • Banners - Helps to add HTML Banner
  • Autocomplete - Setting up auto complete, word lists, styles of search form and its form related configurations are available in this section.

Rules:

  • Query Cleaning -  When the query is hit on the browser, if we need to clean up the query, this rules can be used.
  • Direct Hits -  For a specific query term, if we need to redirect to a specific url, we can do the same here.
  • Pre-search rules - This rules gets executed before the search happens; useful when we need to do an operation on query term(for eg: check query contains a specific word)
  • Post search rules- This gets executed once a search is done and result is available. (An example could be, if a custom variable is present and its value is meeting some criteria)
  • business rules - purely business queries. For eg if business wants to show a banner(promotional Advt) when the query term contains a specific term.
  • Ranking rules - Helps to provide weights to metatags and adjusting its weights. Based on this, search results can be pushed up or down.


Linguistics:

  • Dictionaries - to add our own dictionary.
  • Words & Language -  To add any conditions like case sensitivity, sound alike etc.
  • Did you mean - If a user types a wrong spelled term and if we need to suggest him a relevant term.
  • Excluded words - always helps to omit some terms from search.

Simulator:
An html form of search page in S&P environment. This helps to test the search functionality in Search and Promote itself. We can set up rules for various simulators.

Staging:

A section which gives idea on what are all things modified in stage and not yet pushed to live

Reports:
An over view section of all search operations. Index reports, term reports, Adobe analytics, alerts etc. Changes made by different users can be tracked here.

Index:
All indexing operations can be performed here. This include configuring new index, editing various types of indexes, running index, roll back etc.

Settings:
All account related and user related settings are found here. Crawl, searching, metadata, filtering, rules, analytics, seo, profile, account, user operations etc are managed through thi

s menu.

Accounts:

All accounts are listed here for a logged in user. If the person is a normal user with single account access, only one account will be listed here.