Expat.com is the benchmark platform for people living abroad.
He plays a large role:
- Help futur expats to prepare to settle in their host country;
- Help expats to handle their daily problems;
- Allow members to build networks, both professional and personal in their new place of life;
- Provide content on their new area of residence and on life abroad in general (expatriation guides, magazine dedicated to expatriation);
- Recommend service providers adapted to the needs of those living abroad, list the housing and jobs available in a given geographical area, promote mutual assistance via a collaborative forum and classified ads.
With almost 2.9 million members, research is of utmost importance!
It is mandatory for visitors to find quickly any answer to any question.
In this context, Expat.com made the decision to replace an old Sphinx with Elasticsearch. This allows the company to go from indexing once per day to an almost real-time indexing.
Context of the search project
Expat.com is an SME. So to reduce maintenance time as much as possible, improve time to market and gain flexibility, the choice of the Elastic cloud was quite obvious.
In less than an hour, the company could begin testing in a “High Availability” cluster without any server knowledge.
The content of the website is in 5 languages and we have to handle several data structures:
- Forum
- Articles from a magazine
- City and country guides
- General and real estate classifieds
- Job advertisements
Lot 1 consisted of migrating the site’s search to Elasticsearch:
- Redesign of the indexing procedures
6 search scopes
- Context Suggestions
- General blinded query in every content
- Filtered search by destination and/or by language
- Search in the Forum
- Search in ads
- Results suggestions
Now that the context is in place, let’s give the floor to Christiane Ramiandramanjato, Senior Back-End Web Developer at Expat.com, who gives us her thoughts about the progress of the Elasticsearch implementation project.
Spoon Consulting (SC): Did you know Elasticsearch before this project?
Expat.com : I knew the name of Elasticsearch and roughly its goal. But I really discovered it within the implementation project.
SC: How long did the project take, from its study to the start of production of first part?
Expat.com: The project started about 8 months ago, with a presentation of elasticsearch made by Spoon Consulting.
After it, we took some time to study the project internally.
We did a second workshop to design our indeces based on the recommendations provided by Spoon Consulting.
The 2 key concepts to remember for the modeling of the indices are:
- Dénormalize SQL Data. This was not easy for developers accustomed to relational databases;
- Standardize the fields used to ensure interoperability between indices (make filters on several indices at once).
Once the decision was made, in a few clicks everything was ready.
We first deployed a small test cluster in the Elastic cloud. Before going into production, we upgraded the cluster to be able to withstand the load in High Availability. A few minutes later, the cluster was updated.
SC: What advice would you give to a company starting an search project with Elasticsearch for their website?
Expat.com: First of all, I would advice to well analyze the data structure and especially the use that you want to make of it.
You also need to understand field types usage, this is very important for performance.
Thus, try to adapt the fields mapping as much as possible to the functional need.
Note that, if it is not possible to modify the types of fields of an index after its creation, it’s possible to add additional fields types, or to add new fields without having to index. It will avoids you a lot of reindexing actions.
Finally, consider using index alias. Since indexing is always necessary, using aliases will greatly facilitate your task.
SC: What are the main difficulties you encountered?
Expat.com : Elasticsearch is relativly easy to handle. Creating the first Index requires effort because you have to consider all the links between entities. But once you understood the basics, it becomes much simpler.
The official documentation is very complete and the community is really developed. This is always important when learning a new technology. You will therefore find many examples to guide you.
However, our main request was more complex. The goal was to query on all the indexes of the site by language.
- Forum
- magazine posts
- Country and city guides
- General and real estate classifieds
- Job advertisements
The difficulty was to define a query that works for all cases, without too much boosting one module over the other.
We tended to add a level of complexity every time we wanted to correct a result. But with the help of Spoon consulting, we managed to simplify the query while improving the results.
We continue to work on this query to improve results based on the queries that users do the most.
The other challenging query was the suggestions one.
The principle is very simple in theory, but with a large volume of data, it is difficult to maintain acceptable performance. However, suggestions that take several seconds to appear are completely unnecessary. Applying good practices is therefore essential. On the suggestions, the sanction is immediate.
SC : What will be your next steps with Elasticsearch for Expat.com?
Expat.com: In our first implementation part, we migrated our old system to Elasticsearch. The next step is to take advantage of the tool’s capabilities to develop new modules and offer an enriched experience to our custommers.
We will come back soon to present our use of Elasticsearch in our innovation process.
Spoon consulting is a certified partner of Elastic
As a certified partner of the Elastic company, Spoon Consulting offers a high level consulting for all kinds of companies.
Read more information on your personal use Elasticsearch use case on Spoon consulting’s posts
