compass.utilities.parsing.merge_overlapping_texts

merge_overlapping_texts(text_chunks, n=300)

Merge chunks of text by removing any overlap.

Parameters:

• text_chunks (iterable of str) – Iterable containing text chunks which may or may not contain consecutive overlapping portions.

• n (int, optional) – Number of characters to check at the beginning of each message for overlap with the previous message. Will always be reduced to be less than or equal to half of the length of the previous chunk. By default, 300.

Returns:

str – Merged text.