WEBVTT 00:00:00.700 --> 00:00:05.180 Today we will create an alert file from a small application log. 00:00:05.330 --> 00:00:12.234 The input deliberately includes leading spaces, mixed case, and sentences that mention errors without 00:00:12.234 --> 00:00:13.714 being error records. 00:00:13.864 --> 00:00:21.181 Our goal is to keep real warning and error severity lines, not every sentence containing similar letters. 00:00:21.331 --> 00:00:28.959 Run the supplied script from examples and keep the input log visible beside the generated alerts file as 00:00:28.959 --> 00:00:30.654 we explain each step. 00:00:32.800 --> 00:00:35.211 String functions return new values. 00:00:35.361 --> 00:00:42.781 Trim removes surrounding whitespace, replace changes the exact text we request, and lower converts letter 00:00:42.781 --> 00:00:43.275 case. 00:00:43.425 --> 00:00:47.180 The original binding still refers to the original text. 00:00:47.330 --> 00:00:53.325 This pipeline is a useful way to see how one value moves through several transformations. 00:00:53.475 --> 00:00:59.192 We are demonstrating normalization here, so changing ERROR to WARN is intentional. 00:00:59.342 --> 00:01:04.099 The actual alert extractor keeps the original severity unchanged. 00:01:06.250 --> 00:01:08.405 Text length can mean different things. 00:01:08.555 --> 00:01:15.125 In HHY, length counts Unicode code points, while byte_length counts encoded bytes. 00:01:15.275 --> 00:01:18.902 These two Chinese characters therefore produce two and six. 00:01:19.052 --> 00:01:26.503 A code point count is still not a universal measure of visible screen characters, because some displayed 00:01:26.503 --> 00:01:28.695 symbols combine multiple code points. 00:01:28.845 --> 00:01:36.314 Use the measure that fits your task, and avoid truncating encoded text by treating a byte position as a 00:01:36.314 --> 00:01:37.101 character index. 00:01:39.250 --> 00:01:42.279 Read_lines gives us one line at a time. 00:01:42.429 --> 00:01:48.147 The map callback trims each line before matching, which matters because our pattern will check the 00:01:48.147 --> 00:01:49.576 beginning of the text. 00:01:49.726 --> 00:01:56.031 If we skipped normalization, the leading spaces on the warning record would prevent that anchor from 00:01:56.031 --> 00:01:56.425 matching. 00:01:56.575 --> 00:02:03.993 Keep the order of operations deliberate: first establish the representation you want to inspect, then 00:02:03.993 --> 00:02:07.455 apply the rule to that normalized representation. 00:02:09.600 --> 00:02:13.205 A regex literal places the pattern between slashes. 00:02:13.355 --> 00:02:16.001 The caret anchors this pattern at the beginning. 00:02:16.151 --> 00:02:23.241 The alternatives allow ERROR or WARN, and the word boundary prevents warning from being accepted as the 00:02:23.241 --> 00:02:24.492 shorter WARN token. 00:02:24.642 --> 00:02:26.583 The i flag ignores case. 00:02:26.733 --> 00:02:31.512 Regex_match returns a Boolean, making it suitable for a where predicate. 00:02:31.662 --> 00:02:37.145 Read the pattern aloud as a rule before applying it to an entire input stream. 00:02:39.290 --> 00:02:42.213 Where retains lines whose predicate is true. 00:02:42.363 --> 00:02:48.613 It does not rewrite their content, so our trimmed warning and error messages pass through intact. 00:02:48.763 --> 00:02:53.158 Save_lines then consumes the result and writes the alert file. 00:02:53.308 --> 00:02:56.188 The fixture should produce exactly two lines. 00:02:56.338 --> 00:03:03.314 The informational sentence mentioning errors is excluded, and so is the line beginning with warning. 00:03:03.464 --> 00:03:09.672 Those near misses are useful evidence that our rule is more precise than substring searching. 00:03:11.820 --> 00:03:17.857 Regex_captures is useful when you need details from a match, such as a captured message. 00:03:18.007 --> 00:03:20.909 But when nothing matches, it returns null. 00:03:21.059 --> 00:03:24.621 Check that possibility before accessing capture fields. 00:03:24.771 --> 00:03:31.622 This demonstration intentionally searches an informational line for an error pattern, so the comparison 00:03:31.622 --> 00:03:32.601 prints true. 00:03:32.751 --> 00:03:39.839 Regex_match is simpler when you need only yes or no; choose captures when extracting structured values is 00:03:39.839 --> 00:03:41.924 actually part of the task. 00:03:44.070 --> 00:03:50.449 A broad search for error or warn anywhere in a line would accept both of these distractors. 00:03:50.599 --> 00:03:55.719 That may be right for a full text search, but it is wrong for our severity extractor. 00:03:55.869 --> 00:03:57.895 Requirements decide the pattern. 00:03:58.045 --> 00:04:02.547 Do not make a regex more complicated just to make it look powerful. 00:04:02.697 --> 00:04:09.739 Start with representative positive and negative examples, then verify that your anchors and boundaries 00:04:09.739 --> 00:04:12.254 express the intended log format. 00:04:14.400 --> 00:04:20.629 Add a lowercase error record to the fixture and an informational record that mentions warning. 00:04:20.779 --> 00:04:24.321 Predict which one will enter alerts before running the script. 00:04:24.471 --> 00:04:31.191 Then extend the allowed severity alternatives to include FATAL, and add a corresponding test line. 00:04:31.341 --> 00:04:33.666 Keep one near miss for every new rule. 00:04:33.816 --> 00:04:40.183 You now have a repeatable method for cleaning text, choosing a match predicate, and checking that 00:04:40.183 --> 00:04:42.968 unwanted records stay out of the output.