How To Use ElasticSearch with ApiPlatform

When your SQL Database doesn’t lower it anymore.

Picture by Marten Newhall on Unsplash

The scenario: You have got a working venture and you should add a fulltext search. However most databases (SQL, Doc) don’t include good fuzzy searches, so you should deliver a specialised database (like Elasticsearch) into your venture.

However how?:

You have got a number of methods how one can implement your search database.

1. The connected elastic (for frequent modifications or legacy techniques)

Structure overview (crimson = crucial parts)

Right here, you construct your personal API Platform filter that runs the full-text search on Elasticsearch after which adapts it into an id in (:outcomes) question.

This has some benefits:

  • Legacy Pleasant: You don’t want to alter current filters and privilege checks.
  • Computed properties: you may search computed properties since elasticsearch has the denormalized illustration.
  • Consistency: You all the time fetch the actual knowledge from sql so that you’ll by no means have stale knowledge in your outcomes.

However, this technique has lots of downsides:

  • Efficiency: You all the time must run 2 database connections.
  • Scaling: It’s worthwhile to get all ids from elastic and ahead them to sql.
  • Options: Utilizing superior Elasticsearch options turns into troublesome.
  • Stale Search: You may not discover stale knowledge in Elasticsearch immediately

When to make use of it?

  • If you have already got an API that should not change
  • In case your knowledge modifications often and search index updates can’t sustain

2. The elastic alternative (for read-heavy API’s)

Structure overview (crimson = crucial parts)

Right here, you place the whole lot you want immediately in Elasticsearch. Which means you don’t want your major Database throughout requests anymore.

  • Efficiency, because you retailer the whole lot in Elasticsearch. There may be additionally no overhead for becoming a member of advanced relations anymore. And Elasticsearch can parallelize queries over a number of nodes, making it very quick.
  • Computed properties: you may search computed properties since elasticsearch has the denormalized illustration.
  • Library Help: That is the mannequin some libraries count on you to make use of.

However once more, there are downsides:

  • No write endpoint (API Platform): You lose the aptitude to jot down to that endpoint. You would possibly have the ability to trick API Platform to jot down in SQL and browse in elastic.
  • Legacy Help: You need to (re)write filters and entry rights for Elasticsearch, if you have already got them.
  • Stale Information and Search: In case your replace activity isn’t excellent, you’ll have outdated knowledge in your search outcomes.

When to make use of it?

  • When your database mannequin may be cleanly serialized the best way you want it
  • When your search is a major function and also you wish to use each trick elasticsearch has

3. The elastic addition (versatile, however exhausting to implement)

Structure overview (crimson = crucial parts)

You, once more, put your knowledge immediately into Elasticsearch.

Nonetheless, you construct a separate Mannequin to your API. That manner, it may be one thing utterly completely different than to what you retailer. For instance aggregations, values from information, or exterior API’s.

  • Most Flexibility: You may construct a very completely different illustration than what you’ve got in your database
  • Efficiency, because you retailer the whole lot in Elasticsearch. There may be additionally no overhead for becoming a member of advanced relations anymore.
  • Denormalization Included: In case your knowledge mannequin is advanced or sluggish, then you may simply denormalize into elasticsearch (as an alternative of one other illustration first).
  • Can combine fashions: When you’ve got information, consumer submit and merchandise you wish to search on the similar time, then this structure helps this.

However

  • Effort: You construct a very completely different search mannequin and endpoint.
  • Library Help: For some purpose, there may be barely any library on the market anticipating a separate search mannequin, so you’re on you personal right here.
  • Stale Information and Search: In case your replace activity isn’t excellent, you’ll need to cope with outdated knowledge in your search outcomes.

When to make use of it?

  • when your database mannequin has lots of relations and/or is sluggish
  • if you happen to want a mannequin, that’s utterly completely different to your area mannequin
  • if you wish to put a number of completely different fashions in a single search index

Elasticsearch Variations

It is a resolution you’ll need to make fairly early:

  • Elasticsearch 6.* continues to be essentially the most supported within the PHP world but is EOL. Additionally it is the final model to help unencrypted HTTP, which makes native improvement so much less complicated.
  • Elasticsearch 7.* eliminated sorts and is due to this fact considerably incompatible (the url of the search endpoint doesn’t embrace the sort anymore)
  • Elasticsearch 8.* has principally no help within the php world and has the licensing problem if you happen to use AWS which forked Opensearch.
  • Opensearch is an elasticsearch fork that’s appropriate with model 7.

Elasticsearch 7 is meant to get help till 2023–08–01, so I’d keep on with that. When you plan to make use of AWS, then you may swap to Opensearch.

ApiPlatform Help

ApiPlatform technically has support for elasticsearch. However all API’s are marked as experimental in source code and that’s for a purpose. The ideas aren’t fleshed out. They don’t even support scoring so that you principally can’t construct a full textual content search it…

You might be higher off constructing your personal suppliers and filters, which is definitely quiet easy if in case you have a information (I will present you in the long run)

FOSElasticaBundle

It is a nice bundle to begin your Elasticsearch integration with, particularly to populate your search index.

However truly looking with it is going to trigger you some complications because it needlessly has its personal question language, which you’ll wish to keep away from.

With that out of the best way…

Since I mentioned 3 architectures, I’m solely exhibiting some basic particulars. I construct a number of initiatives with elasticsearch and someway each venture is barely completely different.

Configure Indexing

You may just about comply with the installation instructions of the FOS Elastica Bundle. Outline a serialization group on the properties you wish to index after which configure the Elastica Bundle to make use of that mannequin and serialization group. That is comparable in all 3 architectures.

In case your mannequin is within the doctrine orm, then it’s mechanically listed when up to date. You may set off the primary populate utilizing:

bin/console fos:elastica:populate

When you attempt to implement structure 3., you then’ll must create your personal supplier. There may be solely small documentation for manual-providers. However you’ll principally need to create a pager that gives the objects which are serialized. This supplier can then be configured beneath persistence.supplier. There you are able to do no matter stunt you wish to create your Search Mannequin.

Configure API Platform for Variant 2 and three

API Platform can natively talk to Elasticseach… however solely with model 6 on the time of scripting this. And since there are lots of different limitations as properly, I wouldn’t suggest that implementation.

As an alternative, it is best to write our personal API Platform DataProvider.

This supplier implements 2. and three., as we will reuse current lessons from API Platforms implementation. You may pass over the serializer, if you happen to listed the proper illustration first and secure some efficiency there.

Configure API Platform for variant 1

Right here, you should construct a filter into your current construction mannequin. There are guides on learn how to construct customized filters, like this one:

Your filter will then look one thing like this:

Truly Looking out

You now have the whole lot to combine your fashions into Elasticsearch.
Subsequent, you’ll must construct up your question, however there are sufficient guides on the market, like this one:

I hope this helps you get began in your journey of fine search.

More Posts