Repository navigation
feat(strings): add suffix array algorithm - #15688
hasnain607 wants to merge 5 commits into
Conversation
|
What is the advantage over suffix trees? |
Thanks for the review! The main advantages are memory and simplicity. A suffix array stores one integer per suffix, while a suffix tree needs up to 2n nodes, each holding child pointers |
|
add-suffix-array |
| >>> suffix_array("a") | ||
| [0] | ||
| >>> suffix_array("") | ||
| [] |
There was a problem hiding this comment.
Correctness looks good: I compared this against the naive sorted(range(n), key=lambda i: text[i:]) on 3,000 random strings over small alphabets, including non-ASCII characters, with no mismatches. Could you add a doctest that does the same cross-check, e.g. >>> all(suffix_array(s) == sorted(range(len(s)), key=lambda i: s[i:]) over a few strings like "aaaa", "abab", "banana" and a longer string? That documents the invariant and protects future refactors.
There was a problem hiding this comment.
Added, using sorted(range(len(s)), key=lambda i: s[i:]) over "aaaa", "abab", "banana", "mississippi" and "abracadabra", so the invariant is documented in the doctests.
| new_rank = [0] * n | ||
| for position in range(1, n): | ||
| previous, current = suffixes[position - 1], suffixes[position] | ||
| previous_key = ( |
There was a problem hiding this comment.
The (rank[i], rank[i + k] or -1) pair is built here for previous and current, and again on line 52 for every suffix. Since keys is already sorted in the same order as suffixes, you can compare the first two entries of neighbouring keys instead, which removes lines 61 to 68 and the duplicated -1 handling:
new_rank = [0] * n
for position in range(1, n):
previous, current = suffixes[position - 1], suffixes[position]
same_prefix = keys[position - 1][:2] == keys[position][:2]
new_rank[current] = new_rank[previous] + int(not same_prefix)
There was a problem hiding this comment.
Applied. The renumbering loop now compares keys[position - 1][:2] with keys[position][:2], so the per-suffix pair rebuilding and the duplicated -1 handling are gone.
|
|
||
| Construction here uses prefix doubling (Manber-Myers). | ||
|
|
||
| Time Complexity: O(n log² n) - where n is the length of the text |
There was a problem hiding this comment.
This answers cclauss's question well. Since the advantage over suffix trees is memory and simplicity, please say plainly that construction here is slower than the O(n) approaches (Ukkonen for suffix trees, SA-IS for suffix arrays). Also, "a node per character" (lines 6 to 7) is slightly loose: a suffix tree has up to 2n nodes, so "up to 2n nodes" is more precise.
There was a problem hiding this comment.
Both are in the module docstring now: "up to 2n nodes" for suffix trees, and an explicit sentence that prefix doubling is O(n log² n) versus the O(n) constructions (Ukkonen, SA-IS).
| # pair (rank of the first k, rank of the next k). A second half that | ||
| # runs past the end gets -1: ranks are never negative, so a shorter | ||
| # suffix sorts before a longer one that shares its prefix. | ||
| keys = [(rank[i], rank[i + k] if i + k < n else -1, i) for i in range(n)] |
There was a problem hiding this comment.
Minor: the third tuple element i is only a tie-breaker, and the renumbering step makes tied entries share a rank. A short comment saying so would save readers a moment.
There was a problem hiding this comment.
Added a comment noting the index only makes the sort deterministic and that tied suffixes share a rank in the renumbering step.
| break | ||
| k *= 2 | ||
|
|
||
| return suffixes |
There was a problem hiding this comment.
Optional: most files in this repo end with an if __name__ == "__main__": import doctest; doctest.testmod() block so contributors can run the doctests directly. CI runs them anyway, so this only helps local runs.
There was a problem hiding this comment.
Added the if __name__ == "__main__": block with doctest.testmod().
jgohel9902
left a comment
There was a problem hiding this comment.
Thanks for the contribution. I compared the implementation against a naive sorted(range(n), key=lambda i: text[i:]) on a few thousand random strings, including non-ASCII ones, and found no mismatches. The prefix-doubling logic and the early exit are correct and well commented. My suggestions are small: add a cross-check doctest against the naive version, reuse the already sorted keys instead of rebuilding them in the renumbering loop, and state that construction is slower than the O(n) suffix-tree and SA-IS approaches. Nothing here blocks the PR.
- add a doctest cross-checking the result against a naive sort - compare neighbouring sorted keys when renumbering instead of rebuilding the rank pairs - document the node bound against suffix trees and the O(n log^2 n) cost against linear-time constructions - explain the tie-breaking index - run the doctests under __main__
|
All five review points are addressed in 4bd43e4:
Local verification: doctests pass, 4,000 random strings (including non-ASCII and spaces) match the naive sort with zero mismatches, |
Describe your change
Checklist