org.apache.spark.sql.execution.streaming.FileStreamSource
Add a new file to the map.
Returns true if we should consider this file a new file.
Returns true if we should consider this file a new file. The file is only considered "new" if it is new enough that we are still tracking, and we have not seen it before.
Removes aged entries and returns the number of files removed.
A custom hash map used to track the list of files seen. This map is not thread-safe.
To prevent the hash map from growing indefinitely, a purge function is available to remove files "maxAgeMs" older than the latest file.