355 Commits

Author SHA1 Message Date
Matt W
6a84023548 Prefix tree updates (#931)
* WIP Rename SNTPrefixTree to PrefixTree

* WIP Implement the new PrefixTree and tests

* Add Unit type. Fix build and tests.

* lint

* Make NodeCount accessor for tests

* Updated comments
2022-11-14 13:16:49 +00:00
Matt W
1adb6d2726 Update spool to flush on size thresholds instead of batch counts (#930) 2022-11-03 14:55:51 -04:00
Russell Hancox
5829363733 GUI: Fix EnableSilentMode key (#927) 2022-11-01 10:11:21 -04:00
Matt W
b24e7e42bf Event metrics (#918)
* WIP. Record event count and processing time metrics. Tests don't currently build.

* Updated tests

* Fix field names

* Remove unused target

* formatting

* Cleanup from PR comments
2022-10-28 14:25:07 -04:00
Pete Markowsky
4821ebebd5 Fix: duplicates bug in SNTMetricSet when using multiple fields (#920)
Fix duplicates bug in SNTMetricSet when using multiple fields names.

This also fixes the santactl metric command and golden files for tests.
2022-10-28 13:50:08 -04:00
videlanicolas
3f3de02644 USB: usbBlockMessage is not being used. (#915) 2022-10-26 17:42:49 -04:00
Matt W
899ca89e23 Proto minimization (#909)
* Create Light variants of File and ProcessInfo messages to reduce disk/wire byte counts

* Updated golden test data
2022-10-21 19:48:37 -04:00
Matt W
e7281f1c55 Spool writer (#908)
* Spool writer and santactl command to print proto file

* Make valid JSON for multiple paths. Can now create proto/spool logger. Updated logger tests.

* Make fsspool writer and fsspool log batch writer injectable

* Add spool writer tests

* Updated help text for santactl printlog

* Include file cleanup

* Fix dispatch source destruction

* Change config keys for the new Spool writer

* Spool settings now configurable

* Fix param order

* Remove some test sleeps related to control flow
2022-10-21 16:43:12 -04:00
Matt W
bf0ca24ae7 Machine id proto (#907)
* Add MachineID to all BasicString serialized log messages

* machine_id now a top level proto field

* Remove commented code
2022-10-19 10:51:38 -04:00
np5
4fe8b7908f sync: Fix USB blocking config sync (#890) 2022-10-18 10:01:20 -04:00
Matt W
a8dd332402 Update include paths and add include guard (#905) 2022-10-14 17:58:36 -04:00
Matt W
6631b0a8e3 More import fixes (#904)
* Layering check disable

* workaround for layering issue
2022-10-14 17:20:20 -04:00
Matt W
07e09db608 Import fixes (#902)
* Apply clang-format to cc files

* Modify binaryproto namespace

* Add more required includes

* Add proto includes

* Assert message parsing succeeds in test

* Add optional keyword to proto fields to track presence. TESTS BROKEN.

* Update golden test data
2022-10-14 15:51:53 -04:00
Matt W
1683e09cc8 Proto serializer (#897)
* Initial proto serializer with close event

* Define move ctors for enriched types, delete copy ctors

* More event proto serialization. Commonized proto test code.

* Started work serializing exec event. Added serializer utilities.

* More progress serializing exec event

* Add mroe test data. Test restructure to permit fine grained mocking.

* Env/FD ES types now wrapped in EndpointSecurityAPI. Added calls to proto serializer.

* Add fd type names to proto

* Version compat. Script and Working Dir encoding.

* Add process start time

* Serialize Link event

* Add null check, mainly to fix tests

* Handle versioned expectations

* Each test now build msg in callbacks to set better expectations

* Serialize rename event and tests

* Serialize unlink event and tests

* Serialize allowlist and bundle events. Add utilities tests.

* Formatting

* Disk event proto serialization and tests

* Fix test only issues

* Rename santa_new.proto to santa.proto

* Change fd type int and string to an enum

* Proto namespace now versioned

* Added comments to proto schema

* Add proto support to indicate if fd list truncated
2022-10-13 13:52:41 -04:00
Ivan Tadeu Ferreira Antunes Filho
d6c73e0c6c common: Make SNTCommonEnums a textual header (#896)
This change fixes -wunused-variable warnings. The header is not valid by itself and should be declared as a textual header rather than as a header.
2022-10-03 13:15:33 -04:00
Matt W
7a7f0cd5a8 Ingestion fixups (#891) 2022-09-22 12:30:34 -04:00
Matt W
fcb49701b3 ES and Logging Interfaces Redesign (#888)
* Initial structure for ES wrappers, enriched types, logging

* Basic working ES and logging functionality

* Add in oneTBB and thread-safe-lru deps

* Added a bunch of enriched types

* Auto-mute self when establishing ES client

* Basic auth, tamper client. Syslog of all events. Basic compiler tracking.

* Update copyright header blobs, convert some tabs to spaces

* Auth result cache. Fix getting translocation path.

* Added remaining cache methods

* Add AuthResultCache to Recorder client. Cache now operates on es_file_t.

* Hooked up SNTPrefixTree

* Fix CompilerController for RENAME. Fix AllowList logging missing path.

* Block loading Santa kext

* Added device manager client

* Properly log DiskAppear events

* Fix build to adopt new adhoc build

* Handle clearing cache on UNMOUNT events

* Ignore other ES clients if configured

* Remove SNTAllowlistInfo. Rename AllowList to Allowlist. Minor cleanup.

* Recorder now logs asynchronously. Enricher now returns shared_ptrs.

* Added File writer. Added timestamps to BasicStream serializer.

* Skip calling stat in SNTFileInfo when path given by ES.

* Fix build issue

* Address draft PR feedback

* santactl integrated, XPC works, fix file writer bug

* Integrate syncservice. Start observing some config changes.

* Add metrics service wrapper

* Add metrics config observers and metrics interval reset.

* Start better dependency control. Add Null logger support.

* Added more deps

* Added more deps

* Fix issue where metric service wasn't starting

* Add missing variant include

* Fix missing parent proc name

* Added googletest and new unit test macro

* Started expanding AuthResultCacheTest

* Properly mock EndpointSecurityAPI

* Finished AuthResultCacheTest

* bazelrc now builds all C++ as C++17. Added LoggerTest.

* Add FileTest. Abstract some File constants to Logger.

* Added Empty serializer test

* Started work on BasicStringTest. Fixed some BasicString serialization bugs.

* Added Unlink BasicString serialization test

* Added some more tests. Commonized some test code

* Finished BasicStringTest. Converted to XCTest.

* Standardize esapi variable naming

* Bubble up gTest expect failures to XCTest failures

* AuthResultCacheTest now uses XCTest. Added common TestUtils.h

* EmptyTest now uses XCTest.

* FileTest now uses XCTest

* LoggerTest now uses XCTest. Removed santa_unit_gtest bazel macro.

* Added ClientTest

* Add basic Enricher tests

* Add MessageTest. Make more TestUtils.

* Rename metrics to Metrics

* Add MetricsTest.

* Apply template pattern to Serializer

* Add SNTDecisionCacheTest.

* Add SNTCachedDecisionTest.

* Testing with coveralls debug mode

* Allow manual CI runs

* Remove unused property

* Started work on SNTEndpointSecurityClientTest.

* WIP SNTEndpointSecurityClientTest, fix test run issue

* Added more base ES client tests

* Add more base ES client tests

* Base ES client tests done. Added serializer utils/tests. Expanded basic string tests.

* Add utils test to test suite

* Add copy ctor. Add test output to bazel coverage.

* Single thread bazel coverage

* Updaload coverage file

* Updaload coverage file

* Old gen cov test

* Restructure message handlers to enable better testability

* Added enable tests for all ES clients

* Made a single MockEndpointSecurityAPI class to share everywhere

* Added most of SNTCompilerControllerTest

* Cleanup SNTCompilerControllerTest

* Started expanding Auth client test

* Finished up the Authorizer tests

* Move to using enum class for notify/auth instead of bool

* WIP for tamper resistance test. ASAN issues.

* Add OCMock patch to fix test issue on ARM Macs

* Changed patches directory name to external_patches

* Update WORKSPACE path

* Finished up Tamper Resistance tests

* Finished up Recorder tests.

* Move SNTExecutionControllerTest to ObjC++

* Initial work to port SNTExecutionControllerTest

* Finished porting SNTExecutionControllerTest.

* Added SNTExecutionControllerTest to list of unit tests

* Ported SNTEndpointSecurityDeviceManager.

* Test cleanup, use MockESAPI expectation helpers

* Verify SNTEndpointSecurityDeviceManager expectations differently

* Test cleanup, omit gTest param list where unused

* Log message cleanup

* Rename SNTApplicationTest to santad_test.mm

* Finished porting santad_test, formerly SNTApplicationTest

* Fix SNTEndpointSecurityDeviceManager issues

* Pulled in missed fixes. Updated tests.

* Renamed lowercase filenames to match rest of codebase

* Fix non-static dispatch_once_t, and noisy watching compiler log message

* WIP Started process of removing components no longer used

* WIP Continued process of removing components no longer used

* BUILD file cleanup. Proto warning. Removed unused global

* Rename SNTEventProvider to SNTEndpointSecurityEventHandler

* Rename SNTEndpointSecurityEventHandler protocol

* Remove EnableSysxCache option. Remove --quick flag used during dev.

* Ran testing/fix.sh

* Addmissing param to fix.sh that was omitting .mm files.

* clang-format

* Fix linter: find cmd missing .mm ext, git grep exclude patch files.

* Use MakeESProcess default params in tests

* Move variables to camelCase in objc classes

* More case changes

* Sanitize strings

* Change dispatch queue priorities and standardize daemon queue naming

* Exclude patch files in markdown check

* Ensure string log messages end with newline

* Fix BasicStringTest

* Disable clang-format in code producing different results in local/remote versions

* Moved to using date ranges in copyright notices as per current guidelines

* Update Source/common/SNTConfigurator.h

Suggestion adding whitespace in comment to fix clang-format mangling

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>

* Removed santa_panic macro used in one place

* Updated comment about ES cachability

* Pin oneTBB to specific commit

* Address outstanding WORKSPACE 'canonical reproducible form' messages

* Use string append instead of ostringstream due to benchmark results

* Remove use of freind classes in EnrichedTypes.h

* Added SNTKVOManager, removed observers from SNTConfigurator.

* Fixed SNTEndpointSecurityRecorderTest class name

* Reduce usage of the auto keyword

* Each SNTKVOManager instance now adds its own observer

* Replaced more auto keywords with real types.

* Remove leftover code coverage debugging from ci.yml

* Updated comment

* Memoize SNTFileInfo sha256. Reduce some cache sizes.

* Fix issue checking for translocated paths

* Use more performant NSURL creation method

* Fix lint issue

* Address PR feedback

* Use an array literal for kvo objects

* Fix some clang tidy and import issues

* Replace third party LRU cache with SantaCache for now

* Fix clang tidy issues

* Address PR feedback

* Fix comment typo

Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>

* Added todo for when we adopt macOS 13

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>
2022-09-22 10:18:41 -04:00
Russell Hancox
4436e221df GUI: Add silent mode configuration option. (#871)
When enabled, this option disables *all* GUI notifications from Santa. This is intended for kiosk-style machines where it is not expected for users to _ever_ execute unknown binaries.

Fixes #862
2022-08-11 09:17:07 -04:00
bfreezy
8b55ee4da5 santad: only allow root read+write permissions on sync-state.plist (#858) 2022-07-18 13:32:08 -04:00
Pete Markowsky
234f81ea7c Ensure KVO works for USB config options (#853)
Ensure KVO works for USB config options.
2022-07-15 15:13:55 -04:00
Russell Hancox
743c567bf8 santad: Log team ID in execution logs, where available (#850) 2022-07-15 12:41:56 -04:00
Russell Hancox
21220f1499 santad: Add DisableUnknownEventUpload option. (#852) 2022-07-15 12:30:20 -04:00
Russell Hancox
39f3ffe8fc santactl/status: Fix printing of static rules (#848) 2022-07-15 11:53:38 -04:00
Russell Hancox
fbefbc5910 santasyncservice: Keep XSRF token in memory, don't send to daemon (#851) 2022-07-15 11:52:43 -04:00
Russell Hancox
9db00d143d santad: Improve caching of static rules (#847)
In #846 I forgot that  is only a count of the entries so if the config changes but the number of rules remains the same we would never update the cache. This PR moves the processing of the raw config into the KVO handler code so it is not at all in the hot-path.
2022-07-14 10:50:30 -04:00
Russell Hancox
1cc40d59d8 santad: Allow configuring a static set of rules via configuration profile (#846) 2022-07-13 17:58:13 -04:00
Tom Burgin
64096f5d08 adhoc build and run santa (#840)
* adhoc build and run santa

* fold ci into adhoc

* review updates

Co-authored-by: Tom Burgin <bur@chromium.org>
2022-07-07 17:09:53 -04:00
np5
b7421e4499 Add team ID to synced events (#827) 2022-06-24 20:00:55 +00:00
Russell Hancox
a67801d5ed santactl/status: Remove driver connected, re-org USB blocking status (#826) 2022-06-22 14:59:46 -04:00
Russell Hancox
9099409915 Project: Enable layering check, fix all dependency violations (#811) 2022-05-12 14:26:08 -04:00
Tom Burgin
4540a1c656 SNTConfigurator: remove mutability from sync state dict (#807)
Co-authored-by: Tom Burgin <bur@chromium.org>
2022-05-11 14:35:43 -04:00
Russell Hancox
fdfb00368c GUI: Update keys for EventDetailURL. (#802)
The previous change here (#797) was not backward compatible and would be difficult to roll out. This change restores the previously used key and adds 2 new ones for migration. The previous key is marked deprecated and will be removed in the future.
2022-05-09 13:46:13 -04:00
Russell Hancox
6e22da1d97 santad: Add 'null' event logger. Fixes #754 (#799) 2022-05-06 12:22:04 -04:00
Russell Hancox
1725809335 Add config to allow uploading all events (#800)
* Add config to allow uploading all events

This config can be enabled locally or by a sync server and causes the
client to upload all events, not just those for binaries that are or
would be blocked.

Fixes #689
2022-05-06 11:45:53 -04:00
Russell Hancox
d823028b72 Sync: Add option to enable event upload despite clean sync. (#796)
Related to #789
2022-05-03 15:15:42 -04:00
Russell Hancox
49b2d6e22a GUI: Add %bundle_or_file_sha% translation key (#797)
* GUI: Add %bundle_or_file_sha% translation key

This mimics the current behavior that %file_sha% previously had and
moves %file_sha% to the expected behavior or just showing the file's
SHA.

Related to #795
2022-05-03 14:59:01 -04:00
Tom Burgin
b5ebe1259c syncservice: implementation and migration (#775)
* review updates

* fix test

* review updates

* log level cleanup

Co-authored-by: Tom Burgin <bur@chromium.org>
Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
2022-04-27 14:54:56 -04:00
Russell Hancox
ff3979263e santad: Use TTY path provided by ES (#785) 2022-04-15 12:48:06 -04:00
Walter Lee
601d726fcc Disable layering check for Objective-C (#781) 2022-04-12 09:06:55 -04:00
Matt W
9bca601ce6 Modified build target names for santa proto (#772) 2022-03-25 13:07:57 -04:00
Russell Hancox
676c02626d santactl/metrics: Allow filtering metrics (#763) 2022-03-22 18:12:14 +00:00
Kent Ma
1c04c3a257 Remove code guarded by #ifdef kernel macros (#752)
* Remove code guarded by #ifdef kernel macros
2022-03-15 14:38:40 -04:00
np5
818d3f645f santactl/sync: Add model identifier to preflight request (#751) 2022-03-15 14:24:05 +00:00
Kent Ma
211dbd123f Remove the Santa kernel extension. (#749)
This includes:

* All of the code in Source/santa_driver containing the kernel extension
* The SNTDriverManager event provider
* All workflows in our CI related to testing if the driver builds
* Installation of the driver in install.sh. Note that code uninstalling existing instances of the driver is still intentionally kept present.
* Kernel extension-specific build rules
* Renames SNTKernelCommon to SNTCommon
* Driver version output from santactl version
* The [SNTConfigurator EnableSystemExtension] configuration key
2022-03-14 18:17:02 -04:00
Matt W
c67364fe76 Protobuf support, maildir format logging (#731)
* Initial protobuf support, maildir logging

Fix build issues in the integration test

Deduped some test code

Formatting

Address feedback from draft PR

Removed legacy labels. Updated docs.

Add in metrics. Fix protobuf logging test.

* Now use the Any proto for the LogBatch wrapper

* Changes based on PR feedback

* Added gauge metrics for spool dir

* Formatting

* Add event time to proto

* Fix build issue after rebase

* Update BUILD rules

* Updated language around protobuf logging to mark as beta
2022-03-14 15:46:52 -04:00
Russell Hancox
2f408936a0 Project: Disable bazel layering_check feature for most rules (#742) 2022-03-10 10:07:15 -05:00
Kent Ma
c7e309ccb1 Add a USB device blocking popup. (#728)
* Add a USB device blocking popup.
* Refactor SNTNotificationManager and SNTMessageWindowController to make
  for generalized notification logic
* Add the configuration keys for custom block messages and resize window
2022-02-28 13:30:56 -05:00
Pete Markowsky
faa8946056 Fix: remediate a crash in santametricservice (#729)
* Fix issue with task cancelation.

* Make export timeouts configurable.

This allows an export timeout to be set via configuration and eases testing.
2022-02-14 13:51:29 -05:00
Russell Hancox
58e2b7e1b8 santad: Add fail-closed mode (#722) 2022-01-28 18:29:18 -05:00
Pete Markowsky
cb61d0cc99 Create test suites for each component (#702)
Create test suites for each component.
2022-01-18 17:00:44 -05:00