Full text search ranking out of order

I’m using fulltext search to find the best matching paragraph. I’m getting the matches but the results are out of order.

type Paragraph {
    paragraphId: String! @id
    plainText: String @search(by: [fulltext])
}

Here are the queries and results

I’m searching for the phase “Brutus is an honourable man” across all of Shakespeare’s works.

  1. alloftext query
query MyQuery {
  queryParagraph(filter: {plainText: {alloftext: "Brutus is an honourable man"}}) {
    plainText
  }
}
{
  "data": {
    "queryParagraph": [
      {
        "plainText": "Free from the bondage you are in, Messala:\n[p]The conquerors can but make a fire of him;\n[p]For Brutus only overcame himself,\n[p]And no man else hath honour by his death.\n"
      },
      {
        "plainText": "Good friends, sweet friends, let me not stir you up\n[p]To such a sudden flood of mutiny.\n[p]They that have done this deed are honourable:\n[p]What private griefs they have, alas, I know not,\n[p]That made them do it: they are wise and honourable,\n[p]And will, no doubt, with reasons answer you.\n[p]I come not, friends, to steal away your hearts:\n[p]I am no orator, as Brutus is;\n[p]But, as you know me all, a plain blunt man,\n[p]That love my friend; and that they know full well\n[p]That gave me public leave to speak of him:\n[p]For I have neither wit, nor words, nor worth,\n[p]Action, nor utterance, nor the power of speech,\n[p]To stir men's blood: I only speak right on;\n[p]I tell you that which you yourselves do know;\n[p]Show you sweet Caesar's wounds, poor poor dumb mouths,\n[p]And bid them speak for me: but were I Brutus,\n[p]And Brutus Antony, there were an Antony\n[p]Would ruffle up your spirits and put a tongue\n[p]In every wound of Caesar that should move\n[p]The stones of Rome to rise and mutiny.\n"
      },
      {
        "plainText": "I know that virtue to be in you, Brutus,\n[p]As well as I do know your outward favour.\n[p]Well, honour is the subject of my story.\n[p]I cannot tell what you and other men\n[p]Think of this life; but, for my single self,\n[p]I had as lief not be as live to be\n[p]In awe of such a thing as I myself.\n[p]I was born free as Caesar; so were you:\n[p]We both have fed as well, and we can both\n[p]Endure the winter's cold as well as he:\n[p]For once, upon a raw and gusty day,\n[p]The troubled Tiber chafing with her shores,\n[p]Caesar said to me 'Darest thou, Cassius, now\n[p]Leap in with me into this angry flood,\n[p]And swim to yonder point?' Upon the word,\n[p]Accoutred as I was, I plunged in\n[p]And bade him follow; so indeed he did.\n[p]The torrent roar'd, and we did buffet it\n[p]With lusty sinews, throwing it aside\n[p]And stemming it with hearts of controversy;\n[p]But ere we could arrive the point proposed,\n[p]Caesar cried 'Help me, Cassius, or I sink!'\n[p]I, as Aeneas, our great ancestor,\n[p]Did from the flames of Troy upon his shoulder\n[p]The old Anchises bear, so from the waves of Tiber\n[p]Did I the tired Caesar. And this man\n[p]Is now become a god, and Cassius is\n[p]A wretched creature and must bend his body,\n[p]If Caesar carelessly but nod on him.\n[p]He had a fever when he was in Spain,\n[p]And when the fit was on him, I did mark\n[p]How he did shake: 'tis true, this god did shake;\n[p]His coward lips did from their colour fly,\n[p]And that same eye whose bend doth awe the world\n[p]Did lose his lustre: I did hear him groan:\n[p]Ay, and that tongue of his that bade the Romans\n[p]Mark him and write his speeches in their books,\n[p]Alas, it cried 'Give me some drink, Tintinius,'\n[p]As a sick girl. Ye gods, it doth amaze me\n[p]A man of such a feeble temper should\n[p]So get the start of the majestic world\n[p]And bear the palm alone.\n"
      },
      {
        "plainText": "Friends, Romans, countrymen, lend me your ears;\n[p]I come to bury Caesar, not to praise him.\n[p]The evil that men do lives after them;\n[p]The good is oft interred with their bones;\n[p]So let it be with Caesar. The noble Brutus\n[p]Hath told you Caesar was ambitious:\n[p]If it were so, it was a grievous fault,\n[p]And grievously hath Caesar answer'd it.\n[p]Here, under leave of Brutus and the rest--\n[p]For Brutus is an honourable man;\n[p]So are they all, all honourable men--\n[p]Come I to speak in Caesar's funeral.\n[p]He was my friend, faithful and just to me:\n[p]But Brutus says he was ambitious;\n[p]And Brutus is an honourable man.\n[p]He hath brought many captives home to Rome\n[p]Whose ransoms did the general coffers fill:\n[p]Did this in Caesar seem ambitious?\n[p]When that the poor have cried, Caesar hath wept:\n[p]Ambition should be made of sterner stuff:\n[p]Yet Brutus says he was ambitious;\n[p]And Brutus is an honourable man.\n[p]You all did see that on the Lupercal\n[p]I thrice presented him a kingly crown,\n[p]Which he did thrice refuse: was this ambition?\n[p]Yet Brutus says he was ambitious;\n[p]And, sure, he is an honourable man.\n[p]I speak not to disprove what Brutus spoke,\n[p]But here I am to speak what I do know.\n[p]You all did love him once, not without cause:\n[p]What cause withholds you then, to mourn for him?\n[p]O judgment! thou art fled to brutish beasts,\n[p]And men have lost their reason. Bear with me;\n[p]My heart is in the coffin there with Caesar,\n[p]And I must pause till it come back to me.\n"
      }
    ]
  },
  "extensions": {
    "touched_uids": 12,
    "tracing": {
      "version": 1,
      "startTime": "2020-10-27T18:09:58.624899053Z",
      "endTime": "2020-10-27T18:09:58.630262676Z",
      "duration": 5363643,
      "execution": {
        "resolvers": [
          {
            "path": [
              "queryParagraph"
            ],
            "parentType": "Query",
            "fieldName": "queryParagraph",
            "returnType": "[Paragraph]",
            "startOffset": 108347,
            "duration": 5220953,
            "dgraph": [
              {
                "label": "query",
                "startOffset": 149633,
                "duration": 5117778
              }
            ]
          }
        ]
      }
    },
    "queryCost": 1
  }
}
  1. anyoftext query: It returns around a hundred results.

The phase “Brutus is an honourable man” is clearly present in the 4th result. I had hoped that this result would be ranked first.

Filtering is separate from ordering. At the moment, there is no way to order based on best match. Can you do it client side?

This is correct. I recently created a search engine backed by Slash GraphQL. If you need help wrt ranking results, let me know.

@chewxy, I am interested if you accomplished ordering by sub fields or filtering by sub fields. Do you have a repo link?

sadly not really. The full graphql query looks something like this:

query Search($q: String!) {
  queryPost(filter: {body: {anyoftext: $q}, or: { title: {anyoftext: $q}}}) {
    body
    title
    created_at
    topic_id
    post_id
    pr
  }
}

The body and title are then passed thru a BM25 algorithm to create an initial ranking. The BM25 scores are then multiplied with the pre-computed pagerank. The results are then filtered such that one topic_id only yields a maximum of N results in the final result. Finally it’s sorted according to the sort criteria

1 Like

Here’s a client side ranking solution using Lunr.js

const WHO_SAID_QUERY = gql`
    query WhoSaid($phase: String) {
        queryParagraph(filter: {plainText: {alloftext: $phase}}) {
            plainText
            character {
                charName
            }
            work {
                title
            }
        }
    }`;

const response = await client.query({ query: WHO_SAID_QUERY, variables: { phase } });
const paragraphs = response.data.queryParagraph;

var idx = lunr(function () {
    this.ref('position')
    this.field('text')

    paragraphs.forEach((paragraph, position) => {
        const item = {
            position: position.toString(),
            text: paragraph.plainText
        };

        this.add(item);
    });
});
const searchResult = idx.search(phase);