Commit Graph

539 Commits

Author SHA1 Message Date
Seednode 7546c6257f Add garbage collection after every index rebuild, to test something 2024-01-14 10:12:24 -06:00
Seednode ee09812376 Remove unused CompressionFormats variable 2024-01-14 10:00:34 -06:00
Seednode 6bd97f30c2 Re-use zstd encoder 2024-01-14 09:43:22 -06:00
Seednode 2f06ae3605 Remove non-zstd compression options 2024-01-14 09:29:22 -06:00
Seednode 7868f0d7ad Register all pprof endpoints 2024-01-14 09:06:22 -06:00
Seednode 4e85621335 Round all durations to microseconds 2024-01-11 03:36:16 -06:00
Seednode e3e6aca2fa Set default concurrency back to 10240, as values above inode limits(?) cause panics currently 2024-01-11 03:21:10 -06:00
Seednode ebb5e48d9a Move regex compilation to global variable 2024-01-10 13:07:50 -06:00
Seednode 9e458aa0bb Move regex compilation to global variable 2024-01-10 13:07:24 -06:00
Seednode d568d10e78 Move walkPath() defers out to calling function, for more intuitive readability 2024-01-10 11:58:55 -06:00
Seednode 9b6d9464e6 Replace microsecond delay with another sync.WaitGroup 2024-01-09 17:36:25 -06:00
Seednode 6074b7e546 Re-add microsecond delay to walkPath() invocations 2024-01-09 17:17:26 -06:00
Seednode 431de92bbb Remove snappy, lz4, lzw, deflate, and gzip compression options 2024-01-09 13:39:08 -06:00
Seednode 656c74b883 Change lzw from LSB-first to MSB-first. This should probably be a major version bump, but who's using lzw, really 2024-01-08 17:42:13 -06:00
Seednode 37436377cf Updated transitive dependencies 2024-01-08 16:04:31 -06:00
Seednode e13c141c5c Move compression format slice variable from root.go => index.go 2024-01-08 16:00:37 -06:00
Seednode 56c6565cb1 Add error handling to all functions that can possibly return non-nil errors 2024-01-07 22:43:51 -06:00
Seednode 09bff1ed18 Skip calling IOReadCloser() method on zstd decoder for getReader() 2024-01-07 22:28:03 -06:00
Seednode de94c5eb49 Redact admin prefix when registering profiling handlers, too 2024-01-07 22:25:00 -06:00
Seednode adaa7378ff Updated transitive dependencies 2024-01-07 15:14:37 -06:00
Seednode 787d3880ec Fix typo (lz5 => lz4) in compression format list, change http server error output for consistency with other errors 2024-01-06 09:52:38 -06:00
Seednode 1f6a06bc2c Change default concurrency from max int64 to math.MaxInt32 2024-01-06 09:48:01 -06:00
Seednode f0378f09e0 Change default concurrency from max int64 to math.MaxInt32 2024-01-06 09:47:07 -06:00
Seednode 6f29f89acf Continue scanning instead of aborting on fs.ErrPermission or fs.ErrNotExist; add --debug flag to view output from those errors; initialize error logging earlier in setup 2024-01-06 09:41:30 -06:00
Seednode e9d75ac6f3 Add note that compression can be disabled 2024-01-05 22:17:57 -06:00
Seednode 4403785899 Add missing compression formats to README 2024-01-05 21:55:22 -06:00
Seednode 560868ab75 Remove import alias for pierrec/lz4 2024-01-05 21:48:40 -06:00
Seednode 59cbd25e7c Add --compression-fast to use fastest compression method instead of best 2024-01-05 21:42:22 -06:00
Seednode 965d3deb8d Add support for snappy and lz4 compression 2024-01-05 21:30:08 -06:00
Seednode 2b6aa85302 Add info about compression algorithms and admin prefix redaction to README 2024-01-05 20:42:12 -06:00
Seednode fac4412c5c Add --compression, to allow specifying compression format for index file 2024-01-05 20:39:05 -06:00
Seednode 646c962e5d Add 1 microsecond delay to each call to walkPath() to fix incorrect filesMatched stat 2024-01-05 20:03:49 -06:00
Seednode 60ec29612b Add --redact option to hide admin prefix in log output 2024-01-05 19:00:15 -06:00
Seednode 5167ff457e Allow empty paths, notifying if no valid files are found 2024-01-05 18:50:45 -06:00
Seednode ccff56d28d Rename scanStatsChannel => scanStats, add workaround for occasionally missing single file when indexing 2024-01-05 07:29:16 -06:00
Seednode 89637c12b2 Close zstd encoder before reading file info, to ensure accurate file size reporting for index file 2024-01-04 12:43:26 -06:00
Seednode b90f6cfc64 Updated transitive dependencies 2024-01-04 08:43:52 -06:00
Seednode 9e2b4ad9c3 Add --binary-prefix to output IEC binary prefixes instead of SI decimal prefixes 2024-01-03 12:37:32 -06:00
Seednode f5dae78483 Update README with new default value for --concurrency 2024-01-01 12:05:43 -06:00
Seednode 01032aaab9 Remove max concurrency hard limit 2024-01-01 11:51:28 -06:00
Seednode e032be0936 Set default concurrency from 1024->8192, replace chan int with chan struct{} for concurrency limits 2024-01-01 11:36:19 -06:00
Seednode 127e0ffe22 Restrict --admin-prefix values to same character set as --ignore-file 2023-12-31 16:04:54 -06:00
Seednode 3702cac56e Properly escape '-' in ignore filename pattern 2023-12-31 15:57:39 -06:00
Seednode 843e54c36a Add -_ to supported character list for ignore filename 2023-12-31 15:54:43 -06:00
Seednode 23fe33fbd3 Add support for configurable ignore-file name 2023-12-31 15:50:13 -06:00
Seednode afd2c8acd7 Updated transitive dependencies 2023-12-27 20:31:18 -06:00
Seednode 75abffda62 Add admin prefix to first/prev/next/last links in info pages 2023-12-22 10:22:42 -06:00
Seednode 01191f7443 Sort configuration options for rootCmd 2023-12-17 09:07:41 -06:00
Seednode c1be0efc4b Add disclaimer to --admin-prefix documentation 2023-12-17 07:28:21 -06:00
Seednode f3ee295663 Update README with --admin-prefix info 2023-12-17 06:50:55 -06:00