388 Commits

Author SHA1 Message Date
Pete Markowsky
7fc06ea9d8 Make the sync client content encoding a tunable (#1076)
Make the sync client content encoding a tunable.

This makes the sync client's content encoding a tunable so that it can be
compatible with more sync servers.

Removed the "backwards compatibility" config option.

---------

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
2023-04-24 15:00:29 +02:00
Russell Hancox
46efd6893f config: Add EnableSilentTTYMode key to disable TTY notifications. (#1072)
Fixes #1067
2023-04-19 10:38:24 -04:00
Matt W
50232578d6 Fix string length issues (#1070) 2023-04-13 10:03:52 -04:00
Matt W
be87b3eaf2 Change types of repeated args and envs fields (#1063)
* Change types of repeated args and envs fields

* Update args and env testdata strings to base64

* Remove whitespace
2023-03-31 13:18:09 -04:00
Russell Hancox
c3b2fbf512 sync: Allow server to override the header for transmitting XSRF tokens (#1060)
This change allows a sync server to change the header that Santa will use to send XSRF tokens on subsequent requests by putting the header name in the  header.
2023-03-27 18:11:11 -04:00
Nick Gregory
5295faef0e Fix a couple last TSAN failures (#1056)
* Skip testHandleMessage when testing with tsan

* fix other 2 tsan failures

* change action_env->test_env in bazelrc for sanitizers

* revert Source/santactl/BUILD formatting
2023-03-23 11:11:29 -04:00
Matt W
53ca5eb811 Support filesystem monitoring config embedded in main Santa config (#1054)
* Allow setting file access policy in main Santa config

* Add some tests
2023-03-20 16:47:34 -04:00
Matt W
33c7aab9f1 Basic rate limiting for File Access Authorizer (#1053)
* WIP basic rate limiting support

* WIP added basic metrics when rate limited

* Hookup new metrics

* Cleanup old TODO

* PR feedback, update comments
2023-03-20 15:58:49 -04:00
Russell Hancox
8055b451bb Config: Ignore static rules with an invalid identifier (#1049) 2023-03-07 10:33:13 -05:00
Matt W
29a50f072c Report log type in santactl status (#1036)
* Report log type in santactl status

* Remove unnecessary fallback case
2023-02-06 14:59:42 -05:00
Matt W
a97e82e316 Replace SNTDecisionCache dictionary with SantaCache (#1034)
* Replace SNTDecisionCache dictionary with SantaCache

* PR feedback. Fix tests.
2023-02-03 15:58:53 -05:00
Russell Hancox
532120ac02 Configurator: Return an unsafe_unretained pointer to avoid needless retain/release (#1035) 2023-02-03 15:55:15 -05:00
Russell Hancox
414fbff721 Project: Fix module maps for swift libraries and their dependencies (#1014) 2023-01-26 09:15:30 -05:00
Matt W
f8d1b2e880 Reduce proto warning severity (#1012) 2023-01-25 14:37:00 -05:00
Russell Hancox
dcbbc33e5e Revert "Configurator: Apply config updates in non-daemon processes (#1003)" (#1008)
This reverts commit 1e88b88ee6.
2023-01-20 13:30:06 -05:00
Russell Hancox
1e88b88ee6 Configurator: Apply config updates in non-daemon processes (#1003) 2023-01-18 10:00:39 -05:00
Matt W
3a3564f36b Add watch item state to santactl status (#1000)
* Add method to get WatchItems state

* Update santactl status with watch items state

* Update status label

* PR feedback - add missing dispatch_group_leave
2023-01-12 10:38:12 -05:00
Matt W
df3aac5baf Change name of santa config keys for file access monitoring (#995) 2023-01-09 21:08:57 -05:00
Matt W
4adad2ecfa More event type support (#992)
* Add truncate and create support

* Add metrics support
2023-01-06 12:51:40 -05:00
Nick Gregory
a2f8030482 Fuzz embedded plist reading (#990)
* fuzz embedded plist reading

* remove newline

* consolidate size checking

* brackets

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
2023-01-03 14:22:31 -05:00
Matt W
338a4f738f Opportunistically use ES cache when possible (#989)
* WIP fixing up ES cacheability in file access client

* Removed old code from before simplification

* Add more tests
2023-01-03 14:09:21 -05:00
Nick Gregory
ca81270bff Fix SNTFileInfo Fuzzing (#985)
* fix SNTFileInfo fd leak

* auto poweroff VM after fuzzing

* lint
2022-12-27 16:20:32 -05:00
Nick Gregory
9423beecc8 fix spinloop when no override config is specified (#981) 2022-12-22 13:51:46 -05:00
Pete Markowsky
b18d4a0e30 Fix SNTFileInfoTest for macOS 13 (#977)
* Fix SNTFileInfoTest for macOS 13
2022-12-22 10:37:22 -05:00
Matt W
60f53bc20a Adopt new ES APIs to monitor target paths (#975)
* WIP begin adopting new ES APIs inverting target mute paths

* Track subscription status so as not to unnecessarily enable/disable

* Properly chain call to invert target mute paths. Fix using wrong Message obj.

* Add base client tests

* Support compiling on older platforms

* More changes to support compiling on older platforms

* Only enable watch items periodic task on macOS 13

* Add more asserts to test

* Disable ES caching for now

* lint
2022-12-20 21:15:01 -05:00
Matt W
ec5e8177fb Serialize File Access events (#964)
* WIP skeleton code for file access event serialization

* Added basic string serializer for file access event

* Added proto string serializer for file access event
2022-12-14 11:04:37 -05:00
Matt W
db6c14ea10 Enrich file access events, prepare for logging (#962)
* WIP refactor file access class to setup logging

* Combined GetPathTarget1 and 2, added some tests.

* Change method name to not be abbrv.

* Remove unnecessary includes

* PR feedback: fix missing path sep, add comments

* Fix test issue
2022-12-12 16:37:47 -05:00
Russell Hancox
958ef52698 Config: In debug builds, allow config to be overriden from a plist file. (#957) 2022-12-08 15:07:59 -05:00
Matt W
d19343bccd Draft proto for new FileAccess log (#952)
* Draft proto for new FileAccess log

* Update Source/common/santa.proto

Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>

Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>
2022-12-07 13:06:47 -05:00
Matt W
09cd78d756 Initial work for File Access Authorizer Client (#949)
* WIP Initial work for new fs watcher client

* WIP basic working mechanics of applying policy to OPEN events

* WIP now support allowing access based on cdhash

* WIP lint fix

* WIP check instigator cdhash and cert hash against policy

* WIP Fix test issue in base ES client class

* WIP Fix test issue in water items test

* Added secondary lookup cache for cert hashes and fallback lookups

* Adopt new SantaVnode name

* Adopt min macOS 11. Adopt new SantaCacheHasher for SantaVnode.

* Rename the es client to FileAccessAuthorizer

* Added some more tests

* Added MockLogger and a lot more tests.

* Removed currently unused subscriptions. Don't enable FS client by default

* lint

* lint after rebase

* Use strtoul for hex string conversion. Update comments.

* PR feedback
2022-12-06 19:52:32 -05:00
Matt W
194a3a6d4a Remove SNTCommon (#945)
* Move santa_action_t to SNTCommonEnums and rename to SNTAction

* Move likely and unlikely macros to a new BranchPrediction header

* Remove SNTCommon.h. Move SantaVnode to its own header.

* Add SantaVnodeHash

* Fix build deps
2022-12-01 09:14:54 -05:00
Matt W
9ff2f0d631 Swtich from task_info to libproc for system resource info (#939)
* Swtich from task_info to libproc for system resource info

* Fix return value

* Convert nanos to seconds

* Make GetTimebase static. Expose NanosToMachTime.

* Abstract return or GetTaskInfo to new type.
2022-11-29 16:50:37 -05:00
Matt W
85058ec290 Rename santa_vnode_id_t to SantaVnode (#943)
* Rename santa_vnode_id_t to SantaVnode. Add factory.

* Change types of SantaVnode to match stat(2)
2022-11-28 23:45:14 -05:00
Matt W
6a84023548 Prefix tree updates (#931)
* WIP Rename SNTPrefixTree to PrefixTree

* WIP Implement the new PrefixTree and tests

* Add Unit type. Fix build and tests.

* lint

* Make NodeCount accessor for tests

* Updated comments
2022-11-14 13:16:49 +00:00
Matt W
1adb6d2726 Update spool to flush on size thresholds instead of batch counts (#930) 2022-11-03 14:55:51 -04:00
Russell Hancox
5829363733 GUI: Fix EnableSilentMode key (#927) 2022-11-01 10:11:21 -04:00
Matt W
b24e7e42bf Event metrics (#918)
* WIP. Record event count and processing time metrics. Tests don't currently build.

* Updated tests

* Fix field names

* Remove unused target

* formatting

* Cleanup from PR comments
2022-10-28 14:25:07 -04:00
Pete Markowsky
4821ebebd5 Fix: duplicates bug in SNTMetricSet when using multiple fields (#920)
Fix duplicates bug in SNTMetricSet when using multiple fields names.

This also fixes the santactl metric command and golden files for tests.
2022-10-28 13:50:08 -04:00
videlanicolas
3f3de02644 USB: usbBlockMessage is not being used. (#915) 2022-10-26 17:42:49 -04:00
Matt W
899ca89e23 Proto minimization (#909)
* Create Light variants of File and ProcessInfo messages to reduce disk/wire byte counts

* Updated golden test data
2022-10-21 19:48:37 -04:00
Matt W
e7281f1c55 Spool writer (#908)
* Spool writer and santactl command to print proto file

* Make valid JSON for multiple paths. Can now create proto/spool logger. Updated logger tests.

* Make fsspool writer and fsspool log batch writer injectable

* Add spool writer tests

* Updated help text for santactl printlog

* Include file cleanup

* Fix dispatch source destruction

* Change config keys for the new Spool writer

* Spool settings now configurable

* Fix param order

* Remove some test sleeps related to control flow
2022-10-21 16:43:12 -04:00
Matt W
bf0ca24ae7 Machine id proto (#907)
* Add MachineID to all BasicString serialized log messages

* machine_id now a top level proto field

* Remove commented code
2022-10-19 10:51:38 -04:00
np5
4fe8b7908f sync: Fix USB blocking config sync (#890) 2022-10-18 10:01:20 -04:00
Matt W
a8dd332402 Update include paths and add include guard (#905) 2022-10-14 17:58:36 -04:00
Matt W
6631b0a8e3 More import fixes (#904)
* Layering check disable

* workaround for layering issue
2022-10-14 17:20:20 -04:00
Matt W
07e09db608 Import fixes (#902)
* Apply clang-format to cc files

* Modify binaryproto namespace

* Add more required includes

* Add proto includes

* Assert message parsing succeeds in test

* Add optional keyword to proto fields to track presence. TESTS BROKEN.

* Update golden test data
2022-10-14 15:51:53 -04:00
Matt W
1683e09cc8 Proto serializer (#897)
* Initial proto serializer with close event

* Define move ctors for enriched types, delete copy ctors

* More event proto serialization. Commonized proto test code.

* Started work serializing exec event. Added serializer utilities.

* More progress serializing exec event

* Add mroe test data. Test restructure to permit fine grained mocking.

* Env/FD ES types now wrapped in EndpointSecurityAPI. Added calls to proto serializer.

* Add fd type names to proto

* Version compat. Script and Working Dir encoding.

* Add process start time

* Serialize Link event

* Add null check, mainly to fix tests

* Handle versioned expectations

* Each test now build msg in callbacks to set better expectations

* Serialize rename event and tests

* Serialize unlink event and tests

* Serialize allowlist and bundle events. Add utilities tests.

* Formatting

* Disk event proto serialization and tests

* Fix test only issues

* Rename santa_new.proto to santa.proto

* Change fd type int and string to an enum

* Proto namespace now versioned

* Added comments to proto schema

* Add proto support to indicate if fd list truncated
2022-10-13 13:52:41 -04:00
Ivan Tadeu Ferreira Antunes Filho
d6c73e0c6c common: Make SNTCommonEnums a textual header (#896)
This change fixes -wunused-variable warnings. The header is not valid by itself and should be declared as a textual header rather than as a header.
2022-10-03 13:15:33 -04:00
Matt W
7a7f0cd5a8 Ingestion fixups (#891) 2022-09-22 12:30:34 -04:00
Matt W
fcb49701b3 ES and Logging Interfaces Redesign (#888)
* Initial structure for ES wrappers, enriched types, logging

* Basic working ES and logging functionality

* Add in oneTBB and thread-safe-lru deps

* Added a bunch of enriched types

* Auto-mute self when establishing ES client

* Basic auth, tamper client. Syslog of all events. Basic compiler tracking.

* Update copyright header blobs, convert some tabs to spaces

* Auth result cache. Fix getting translocation path.

* Added remaining cache methods

* Add AuthResultCache to Recorder client. Cache now operates on es_file_t.

* Hooked up SNTPrefixTree

* Fix CompilerController for RENAME. Fix AllowList logging missing path.

* Block loading Santa kext

* Added device manager client

* Properly log DiskAppear events

* Fix build to adopt new adhoc build

* Handle clearing cache on UNMOUNT events

* Ignore other ES clients if configured

* Remove SNTAllowlistInfo. Rename AllowList to Allowlist. Minor cleanup.

* Recorder now logs asynchronously. Enricher now returns shared_ptrs.

* Added File writer. Added timestamps to BasicStream serializer.

* Skip calling stat in SNTFileInfo when path given by ES.

* Fix build issue

* Address draft PR feedback

* santactl integrated, XPC works, fix file writer bug

* Integrate syncservice. Start observing some config changes.

* Add metrics service wrapper

* Add metrics config observers and metrics interval reset.

* Start better dependency control. Add Null logger support.

* Added more deps

* Added more deps

* Fix issue where metric service wasn't starting

* Add missing variant include

* Fix missing parent proc name

* Added googletest and new unit test macro

* Started expanding AuthResultCacheTest

* Properly mock EndpointSecurityAPI

* Finished AuthResultCacheTest

* bazelrc now builds all C++ as C++17. Added LoggerTest.

* Add FileTest. Abstract some File constants to Logger.

* Added Empty serializer test

* Started work on BasicStringTest. Fixed some BasicString serialization bugs.

* Added Unlink BasicString serialization test

* Added some more tests. Commonized some test code

* Finished BasicStringTest. Converted to XCTest.

* Standardize esapi variable naming

* Bubble up gTest expect failures to XCTest failures

* AuthResultCacheTest now uses XCTest. Added common TestUtils.h

* EmptyTest now uses XCTest.

* FileTest now uses XCTest

* LoggerTest now uses XCTest. Removed santa_unit_gtest bazel macro.

* Added ClientTest

* Add basic Enricher tests

* Add MessageTest. Make more TestUtils.

* Rename metrics to Metrics

* Add MetricsTest.

* Apply template pattern to Serializer

* Add SNTDecisionCacheTest.

* Add SNTCachedDecisionTest.

* Testing with coveralls debug mode

* Allow manual CI runs

* Remove unused property

* Started work on SNTEndpointSecurityClientTest.

* WIP SNTEndpointSecurityClientTest, fix test run issue

* Added more base ES client tests

* Add more base ES client tests

* Base ES client tests done. Added serializer utils/tests. Expanded basic string tests.

* Add utils test to test suite

* Add copy ctor. Add test output to bazel coverage.

* Single thread bazel coverage

* Updaload coverage file

* Updaload coverage file

* Old gen cov test

* Restructure message handlers to enable better testability

* Added enable tests for all ES clients

* Made a single MockEndpointSecurityAPI class to share everywhere

* Added most of SNTCompilerControllerTest

* Cleanup SNTCompilerControllerTest

* Started expanding Auth client test

* Finished up the Authorizer tests

* Move to using enum class for notify/auth instead of bool

* WIP for tamper resistance test. ASAN issues.

* Add OCMock patch to fix test issue on ARM Macs

* Changed patches directory name to external_patches

* Update WORKSPACE path

* Finished up Tamper Resistance tests

* Finished up Recorder tests.

* Move SNTExecutionControllerTest to ObjC++

* Initial work to port SNTExecutionControllerTest

* Finished porting SNTExecutionControllerTest.

* Added SNTExecutionControllerTest to list of unit tests

* Ported SNTEndpointSecurityDeviceManager.

* Test cleanup, use MockESAPI expectation helpers

* Verify SNTEndpointSecurityDeviceManager expectations differently

* Test cleanup, omit gTest param list where unused

* Log message cleanup

* Rename SNTApplicationTest to santad_test.mm

* Finished porting santad_test, formerly SNTApplicationTest

* Fix SNTEndpointSecurityDeviceManager issues

* Pulled in missed fixes. Updated tests.

* Renamed lowercase filenames to match rest of codebase

* Fix non-static dispatch_once_t, and noisy watching compiler log message

* WIP Started process of removing components no longer used

* WIP Continued process of removing components no longer used

* BUILD file cleanup. Proto warning. Removed unused global

* Rename SNTEventProvider to SNTEndpointSecurityEventHandler

* Rename SNTEndpointSecurityEventHandler protocol

* Remove EnableSysxCache option. Remove --quick flag used during dev.

* Ran testing/fix.sh

* Addmissing param to fix.sh that was omitting .mm files.

* clang-format

* Fix linter: find cmd missing .mm ext, git grep exclude patch files.

* Use MakeESProcess default params in tests

* Move variables to camelCase in objc classes

* More case changes

* Sanitize strings

* Change dispatch queue priorities and standardize daemon queue naming

* Exclude patch files in markdown check

* Ensure string log messages end with newline

* Fix BasicStringTest

* Disable clang-format in code producing different results in local/remote versions

* Moved to using date ranges in copyright notices as per current guidelines

* Update Source/common/SNTConfigurator.h

Suggestion adding whitespace in comment to fix clang-format mangling

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>

* Removed santa_panic macro used in one place

* Updated comment about ES cachability

* Pin oneTBB to specific commit

* Address outstanding WORKSPACE 'canonical reproducible form' messages

* Use string append instead of ostringstream due to benchmark results

* Remove use of freind classes in EnrichedTypes.h

* Added SNTKVOManager, removed observers from SNTConfigurator.

* Fixed SNTEndpointSecurityRecorderTest class name

* Reduce usage of the auto keyword

* Each SNTKVOManager instance now adds its own observer

* Replaced more auto keywords with real types.

* Remove leftover code coverage debugging from ci.yml

* Updated comment

* Memoize SNTFileInfo sha256. Reduce some cache sizes.

* Fix issue checking for translocated paths

* Use more performant NSURL creation method

* Fix lint issue

* Address PR feedback

* Use an array literal for kvo objects

* Fix some clang tidy and import issues

* Replace third party LRU cache with SantaCache for now

* Fix clang tidy issues

* Address PR feedback

* Fix comment typo

Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>

* Added todo for when we adopt macOS 13

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>
2022-09-22 10:18:41 -04:00