How can I correctly prefix a word with “a” and “an”?

by
  1. Download Wikipedia
  2. Unzip it and write a quick filter program that spits out only article text (the download is generally in XML format, along with non-article metadata too).
  3. Find all instances of a(n)…. and make an index on the following word and all of its prefixes (you can use a simple suffixtrie for this). This should be case sensitive, and you’ll need a maximum word-length – 15 letters?
  4. (optional) Discard all those prefixes which occur less than 5 times or where “a” vs. “an” achieves less than 2/3 majority (or some other threshholds – tweak here). Preferably keep the empty prefix to avoid corner-cases.
  5. You can optimize your prefix database by discarding all those prefixes whose parent shares the same “a” or “an” annotation.
  6. When determining whether to use “A” or “AN” find the longest matching prefix, and follow its lead. If you didn’t discard the empty prefix in step 4, then there will always be a matching prefix (namely the empty prefix), otherwise you may need a special case for a completely-non matching string (such input should be very rare).

You probably can’t get much better than this – and it’ll certainly beat most rule-based systems.

Edit: I’ve implemented this in JS/C#. You can try it in your browser, or download the small, reusable javascript implementation it uses. The .NET implementation is package AvsAn on nuget. The implementations are trivial, so it should be easy to port to any other language if necessary.

Turns out the “rules” are quite a bit more complex than I thought:

  • it’s an unanticipated result but it’s a unanimous vote
  • it’s an honest decision but a honeysuckle shrub
  • Symbols: It’s an 0800 number, or an ∞ of oregano.
  • Acronyms: It’s a NASA scientist, but an NSA analyst; a FIAT car but an FAA policy.

…which just goes to underline that a rule based system would be tricky to build!

More Related Contents:

  1. How to write a regular expression in c# to match internal links to articles in Wikipedia? [duplicate]
  2. How to access object reference using string in c# [duplicate]
  3. How can I convert String to Int?
  4. ASP.NET MVC 5 culture in route and url
  5. Why can’t I define a default constructor for a struct in .NET?
  6. How do I extract text that lies between parentheses (round brackets)?
  7. How to rethrow InnerException without losing stack trace in C#?
  8. Setting WPF image source in code
  9. Difference between InvariantCulture and Ordinal string comparison
  10. Does anyone still use [goto] in C# and if so why? [closed]
  11. use of tilde (~) in asp.net path
  12. C# TcpClient: Send serialized objects using separators?
  13. Convert base class to derived class [duplicate]
  14. IEnumerable doesn’t have a Count method
  15. How can I ignore a property when serializing using the DataContractSerializer?
  16. How to create a new language for use in Visual Studio
  17. nullable object must have a value
  18. Detecting idle users in Winforms
  19. Process files concurrently and asynchronously
  20. How to keep a .NET console app running?
  21. How do I list all loaded assemblies?
  22. What is the C# equivalent of NaN or IsNumeric?
  23. DataGridView with Button Control – Delete Row
  24. Could not find installable ISAM
  25. How do I create a DataTable, then add rows to it?
  26. Look if a method is called inside a method using reflection
  27. How can I check the color depth of a Bitmap?
  28. int to hex string
  29. Efficient way to send images via WCF?
  30. Why are WinForms applications STAThread by default?

Leave a Comment