Htykuut

I have a DataFrame like this:

df = pd.DataFrame({'FK':['QFf','TFzs'],'title':['World Fair 2018','ZUGER MESSE']})

I want Elasticsearch to find all the documents with the similar title as the original title in my DataFrame and calculate the score for each match. in matching titles, year is not important. It means world fair 2018 and world fair 2017 are exactly the same. Below is part of my code but it doesn't give me an efficient matching score.

try:

for event in scan(es,

        query={

      "query": {

        "bool": {

          "must": [

           {"query_string": {"default_field": "data.event.title",

           "query": row['title']}},

          ]}},



        if len(raw_results) < 5:

            raw_results.append(event)

        else:    

            break

    events_list = 

    if len (raw_results) > 0:

        for item in raw_results:



            event_id = item['_id']

            record_score = item['_score']

            event_source = item['_source']['data']['event']

            event_source['event_id'] = event_id

            event_source['score'] = record_score

            events_list.append(event_source)



        dframe2 = pd.DataFrame(events_list)  

The output that I get is similar to below:

 FK    title         event_id     title           score

 QFf  World Fair 2018  EfB     world fair         44.2

 QFf  World Fair 2018  n77     world fair         44.1

 QFf  World Fair 2018  5cY     world fair 2017    25.84

 TFzs ZUGER MESSE      NEQ    Styles Messe        20.12

 TFzs ZUGER MESSE      mBc    Hannover Messe      20.06

 TFzs ZUGER MESSE      Y9S    WunderWelten-Messe  20.0  

As shown above, in the output, the score of "world fair 2017" dropped because of the year difference. It seems that the score is also affected by the length of the title. I should run the code on about 6 million documents. I would appreciate if anyone helps me how can I improve the above code?