WEBVTT 00:00:00.700 --> 00:00:07.207 This project brings file discovery, regular expressions, streams, and parallel work together. 00:00:07.357 --> 00:00:14.055 We have two small log files containing ordinary information messages, warnings, and errors. 00:00:14.205 --> 00:00:20.906 Our goal is a report containing only the warning and error lines, with the source filename attached to 00:00:20.906 --> 00:00:21.651 each one. 00:00:21.801 --> 00:00:25.854 The final output should be deterministic and easy to inspect. 00:00:26.004 --> 00:00:32.605 We will build the pipeline around that result, then check both the successful run and a clear command 00:00:32.605 --> 00:00:33.705 line usage error. 00:00:35.860 --> 00:00:42.004 The script accepts two arguments: a directory containing logs and the destination report. 00:00:42.154 --> 00:00:47.487 Keeping those choices outside the source code makes the same script reusable. 00:00:47.637 --> 00:00:53.077 Run the command from the examples directory so the fixture path resolves as shown. 00:00:53.227 --> 00:00:59.286 The input is deliberately small, and the script does not include a minimum file size filter. 00:00:59.436 --> 00:01:06.965 That means every supplied log participates in the lesson, and you can verify every selected and rejected 00:01:06.965 --> 00:01:10.508 line without scrolling through a large production dataset. 00:01:12.660 --> 00:01:16.927 Before accessing either argument, check that both were provided. 00:01:17.077 --> 00:01:23.797 If the count is wrong, print a concise usage message to standard error and exit with code three. 00:01:23.947 --> 00:01:28.213 This is a small decision with a large effect on usability. 00:01:28.363 --> 00:01:34.273 Someone who runs the script incorrectly gets instructions instead of an index error. 00:01:34.423 --> 00:01:41.440 It also makes automation easier, because a caller can distinguish an invalid invocation from successful 00:01:41.440 --> 00:01:47.521 report generation without guessing from the presence or absence of an output file. 00:01:49.670 --> 00:01:55.025 Convert the first argument to a Path, then discover log files recursively. 00:01:55.175 --> 00:02:02.300 Filesystem traversal order is not the report contract, so we sort by filename before processing. 00:02:02.450 --> 00:02:07.015 The explicit ascending option is part of the validated call form. 00:02:07.165 --> 00:02:13.203 For these fixtures, the filenames are unique, which makes the resulting order straightforward. 00:02:13.353 --> 00:02:20.875 If your real directory has repeated basenames in different subdirectories, include a relative path in the 00:02:20.875 --> 00:02:25.577 label and ordering strategy so readers can distinguish those sources. 00:02:27.730 --> 00:02:30.823 Each worker reads one file as a line stream. 00:02:30.973 --> 00:02:37.920 The regular expression matches either ERROR or WARN, and map prefixes the selected line with its 00:02:37.920 --> 00:02:38.355 filename. 00:02:38.505 --> 00:02:43.497 We use explicit string concatenation here, matching the tested examples. 00:02:43.647 --> 00:02:50.576 Collect inside the worker returns a finite list, rather than trying to send an open file stream back 00:02:50.576 --> 00:02:52.116 across the worker boundary. 00:02:52.266 --> 00:02:55.743 The pattern is intentionally simple and case sensitive. 00:02:55.893 --> 00:03:01.696 It can match those words anywhere, so a structured log format may need a stricter expression. 00:03:03.850 --> 00:03:07.370 Parallel returns one result for every input file. 00:03:07.520 --> 00:03:14.553 Because each result is a list of matching lines, the next stage must flatten those lists into a single 00:03:14.553 --> 00:03:14.923 stream. 00:03:15.073 --> 00:03:19.830 Flat map converts each list back into a Stream and emits its lines. 00:03:19.980 --> 00:03:25.100 Save lines then consumes that final stream and writes the destination. 00:03:25.250 --> 00:03:32.054 Forgetting this flattening step is a common structural mistake: a list of lists is not the same data 00:03:32.054 --> 00:03:34.701 shape as a stream of text lines. 00:03:36.850 --> 00:03:39.367 Run the project and open the report. 00:03:39.517 --> 00:03:45.000 The message report saved is useful feedback, but it is not the whole validation. 00:03:45.150 --> 00:03:51.017 Compare the file with the expected three lines: two from API and one from worker. 00:03:51.167 --> 00:03:56.820 Confirm that the information messages are absent and that the file ordering is stable. 00:03:56.970 --> 00:04:01.045 Then run without arguments and observe the usage failure. 00:04:01.195 --> 00:04:08.724 An unreadable file is not silently ignored by this example; the unhandled error should stop the pipeline 00:04:08.724 --> 00:04:12.267 instead of presenting a partial report as complete. 00:04:14.420 --> 00:04:21.311 Add three lines to a fixture: an information message, an uppercase warning, and a lowercase warning. 00:04:21.461 --> 00:04:24.533 Predict which will match before running the script again. 00:04:24.683 --> 00:04:32.212 Then decide whether your intended log format needs case insensitive matching or a more precise severity 00:04:32.212 --> 00:04:32.683 field. 00:04:32.833 --> 00:04:37.782 A useful report begins with a clear definition of what counts as an alert. 00:04:37.932 --> 00:04:45.467 You now have a complete small utility with a command contract, bounded workers, explicit data shapes, and 00:04:45.467 --> 00:04:49.900 output that can be verified independently of the terminal message.