Systemd Journald and audit logging causing journal issues

[Brad Zynda]

On 10/18/2017 07:27 PM, Steve Grubb wrote:
> On Wednesday, October 18, 2017 12:32:15 PM EDT Brad Zynda wrote:
>> On 10/18/2017 12:26 PM, Steve Grubb wrote:
>>> On Wednesday, October 18, 2017 12:13:13 PM EDT Brad Zynda wrote:
>>>> So now you have to comment out a rule at a time and watch for
>>>> usage/count to fall?
>>>
>>> Well, I am certain that commenting out that rule will drop the count. But
>>> the question more is why is that rule being triggered. One thing you
>>> could do is post the rule to the mail list if you think it might be
>>> formed wrong. But you might also want to see whay its being triggered by
>>> doing something like
>>>
>>> ausearch --start today -k perm_mod --raw | aureport --summary --file -i
>>>
>>> ausearch --start today -k perm_mod --raw | aureport --summary -x -i
>>>
>>> ausearch --start today -k perm_mod --raw | aureport --summary --syscall -i
>>>
>>>> Also if I wanted to grep and compare those numbers and alert with an
>>>> email what would be the magic number to threshold with in a gt statement
>>>> (500, 1000, etc.)?
>>>
>>> That's a bit harder. You'd want a sliding window of time. Assuming your
>>> cron job runs once an hour and a US locale, you'd do something like this:
>>>
>>> aureport --start `date -d '1 hour ago' "+%m/%d/%Y %H:%M:%S"` --key
>>> --summary --input-logs
>>>
>>> I don't know what the best threshold would be because its workload
>>> dependent. If you wanted to see things visualized, I'd try playing with
>>> the data in R.
>>>
>>> http://security-plus-data-science.blogspot.com/2017/03/bar-charts.html
>>> http://security-plus-data-science.blogspot.com/2017/03/heatmaps.html
>>>
>>> That assumes you have a recent audit package (2.7 or higher) and RStudio.
>>>
>>> -Steve
>>
>> Here are the rules:
>>
>> grep perm_mod /etc/audit/audit.rules
>> -a always,exit -F arch=b64 -S chmod -S fchmod -S fchmodat -F auid>=1000
>> -F auid!=4294967295 -k perm_mod
>> -a always,exit -F arch=b32 -S chmod -S fchmod -S fchmodat -F auid>=1000
>> -F auid!=4294967295 -k perm_mod
>> -a always,exit -F arch=b64 -S chown -S fchown -S fchownat -S lchown -F
>> auid>=1000 -F auid!=4294967295 -k perm_mod
>> -a always,exit -F arch=b32 -S chown -S fchown -S fchownat -S lchown -F
>> auid>=1000 -F auid!=4294967295 -k perm_mod
>> -a always,exit -F arch=b64 -S setxattr -S lsetxattr -S fsetxattr -S
>> removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F
>> auid!=4294967295 -k perm_mod
>> -a always,exit -F arch=b32 -S setxattr -S lsetxattr -S fsetxattr -S
>> removexattr -S lremovexattr -S fremovexattr -F auid>=1000 -F
>> auid!=4294967295 -k perm_mod
>>
>> grep delete /etc/audit/audit.rules
>> -a always,exit -F arch=b64 -S unlink -S unlinkat -S rename -S renameat
>> -F auid>=1000 -F auid!=4294967295 -k delete
>> -a always,exit -F arch=b32 -S unlink -S unlinkat -S rename -S renameat
>> -F auid>=1000 -F auid!=4294967295 -k delete
>> -a always,exit -F arch=b64 -S init_module -S delete_module -k modules
>> -a always,exit -F arch=b32 -S init_module -S delete_module -k modules
> 
> These rules are well formed. Meaning no obvious holes that would cause 
> unintended events. The other ausearch/aureport commands I gave you should show 
> you what is causing the events and to which files. This information may be 
> used to create some kind of "never" rule that limits what gets audited. If you 
> do create some exclusion rule, it needs to be above the perm_mod rules because 
> audit is a "first match wins" system.
> 
> -Steve
> 
> -Steve
> 
> 

Here is a peak report (at least in the last 24 hours) didnt include the
file summaries as that would make this email HUGE:

Key Summary Report
===========================
total  key
===========================
8170  perm_mod
5390  delete
1073  access
56  time-change
26  session
12  privileged
7  logins


Syscall Summary Report
==========================
total  syscall
==========================
4250  fchmodat
1613  chmod
831  fchmod
521  fchown
97  chown
12  setxattr


Syscall Summary Report
==========================
total  syscall
==========================
2887  unlink
2189  rename
186  unlinkat


so from the list the top 2 in each perm_mod and delete from the above
list seem to be the culprits..

Thanks,
Brad