Apache Lucene is a high performance Java information retrieval library created by Doug Cutting. It provides a simple and powerful API that makes it easier for developers to add search engine capabilities to their applications. Lucene works by applications building an index of their data for efficient lookup, as sequential scanning is too slow for searching. This indexing process converts data into an optimized structure to facilitate rapid searching, though it requires additional storage space.