21 Commits

Author SHA1 Message Date
Matt W
72ed5ee4f9 Drop macos 11 (#1377)
* Drop macOS 11 support

* More changes after rebase, add basic macOS 15 support
2024-06-28 12:58:07 -04:00
Russell Hancox
8e1e155c23 Project: Re-enable layering_checks (#1350) 2024-05-15 14:05:58 -04:00
Russell Hancox
1672e52b7b Project: Disable layering_check in all BUILD files (#1344) 2024-05-09 15:25:19 -04:00
Tom Burgin
0f5e551345 Project: Fix lint.sh to bubble up all errors, switch from pylint to pyink, fix existing lint errors (#1338) 2024-05-06 16:47:32 -04:00
Nick Gregory
51b0f7146d Testing: update E2E to use JIT runners (#1335)
* inject jit runner token into e2e vm

* split out vm updating

* argparse + logging

* restore update/start vm steps

* pr review comments

* rm gcp.json and verify runner sha
2024-05-06 09:10:04 -04:00
Nick Gregory
1596990c65 reorder e2e tests (#1249) 2023-12-04 13:01:30 -05:00
Pete Markowsky
ea7e11fc22 Add Support for CS_INVALIDATED events (#1210)
Add support for logging when codesigning has become invalidated for a process.

This adds support to the Recorder to log when codesigning is invalidated as reported by the Endpoint Security Framework's
ES_EVENT_TYPE_NOTIFY_CS_INVALIDATED event.
2023-11-02 10:04:18 -04:00
Nick Gregory
7530b8f5c1 Add E2E testing for usb (#1214)
* e2e test usb mounting

* no poweroff

* no start

* drive usb via sync server since its up

sudo santactl status

sudo?

* revert nostart/nopoweroff

* bump VMCLI minimum os version
2023-11-01 11:44:00 -04:00
Nick Gregory
28dd6cbaed Enable e2e testing on macOS 14 (#1209)
* e2e for macos 14

* no shutdown

* gh path

* dismiss santa popup after bad binary

* sleep for ui

* re-enable start vm

* re-enable poweroff

* tabs

* ratchet checkout actions in e2e
2023-10-30 17:45:37 -04:00
Matt W
b02336613a Remove references to old EnableSystemExtension config key (#1155) 2023-08-18 11:47:14 -04:00
Russell Hancox
d82e64aa5f Project: Split integration VM license into its own LICENSE file (#1147) 2023-08-08 13:29:06 -04:00
Nick Gregory
ca81270bff Fix SNTFileInfo Fuzzing (#985)
* fix SNTFileInfo fd leak

* auto poweroff VM after fuzzing

* lint
2022-12-27 16:20:32 -05:00
Nick Gregory
57285c48dd use new public api for booting VM into recoveryOS (#983) 2022-12-22 16:27:38 -05:00
Nick Gregory
9d448071f7 Lint the E2E start-vm Python script (#965)
* appease the linter

* add python to the lint script
2022-12-14 17:37:56 -05:00
Nick Gregory
cd6c0e7120 Introduce end-to-end testing (#919)
* initial e2e work

* switch to entitlements property instead of codesignopts hack

* bring moroz patches in

* go ahead and switch to upstream

* lint

* no need to install gcs every time

* codeowners

* add comments

* move to new e2e workflow

* rename e2e workflow
2022-12-14 11:15:55 -05:00
Matt W
e1dc50fb36 Drop macOS 10.15 (#944)
* Drop macOS 10.15 support

* lint
2022-11-29 20:20:48 -05:00
Matt W
fcb49701b3 ES and Logging Interfaces Redesign (#888)
* Initial structure for ES wrappers, enriched types, logging

* Basic working ES and logging functionality

* Add in oneTBB and thread-safe-lru deps

* Added a bunch of enriched types

* Auto-mute self when establishing ES client

* Basic auth, tamper client. Syslog of all events. Basic compiler tracking.

* Update copyright header blobs, convert some tabs to spaces

* Auth result cache. Fix getting translocation path.

* Added remaining cache methods

* Add AuthResultCache to Recorder client. Cache now operates on es_file_t.

* Hooked up SNTPrefixTree

* Fix CompilerController for RENAME. Fix AllowList logging missing path.

* Block loading Santa kext

* Added device manager client

* Properly log DiskAppear events

* Fix build to adopt new adhoc build

* Handle clearing cache on UNMOUNT events

* Ignore other ES clients if configured

* Remove SNTAllowlistInfo. Rename AllowList to Allowlist. Minor cleanup.

* Recorder now logs asynchronously. Enricher now returns shared_ptrs.

* Added File writer. Added timestamps to BasicStream serializer.

* Skip calling stat in SNTFileInfo when path given by ES.

* Fix build issue

* Address draft PR feedback

* santactl integrated, XPC works, fix file writer bug

* Integrate syncservice. Start observing some config changes.

* Add metrics service wrapper

* Add metrics config observers and metrics interval reset.

* Start better dependency control. Add Null logger support.

* Added more deps

* Added more deps

* Fix issue where metric service wasn't starting

* Add missing variant include

* Fix missing parent proc name

* Added googletest and new unit test macro

* Started expanding AuthResultCacheTest

* Properly mock EndpointSecurityAPI

* Finished AuthResultCacheTest

* bazelrc now builds all C++ as C++17. Added LoggerTest.

* Add FileTest. Abstract some File constants to Logger.

* Added Empty serializer test

* Started work on BasicStringTest. Fixed some BasicString serialization bugs.

* Added Unlink BasicString serialization test

* Added some more tests. Commonized some test code

* Finished BasicStringTest. Converted to XCTest.

* Standardize esapi variable naming

* Bubble up gTest expect failures to XCTest failures

* AuthResultCacheTest now uses XCTest. Added common TestUtils.h

* EmptyTest now uses XCTest.

* FileTest now uses XCTest

* LoggerTest now uses XCTest. Removed santa_unit_gtest bazel macro.

* Added ClientTest

* Add basic Enricher tests

* Add MessageTest. Make more TestUtils.

* Rename metrics to Metrics

* Add MetricsTest.

* Apply template pattern to Serializer

* Add SNTDecisionCacheTest.

* Add SNTCachedDecisionTest.

* Testing with coveralls debug mode

* Allow manual CI runs

* Remove unused property

* Started work on SNTEndpointSecurityClientTest.

* WIP SNTEndpointSecurityClientTest, fix test run issue

* Added more base ES client tests

* Add more base ES client tests

* Base ES client tests done. Added serializer utils/tests. Expanded basic string tests.

* Add utils test to test suite

* Add copy ctor. Add test output to bazel coverage.

* Single thread bazel coverage

* Updaload coverage file

* Updaload coverage file

* Old gen cov test

* Restructure message handlers to enable better testability

* Added enable tests for all ES clients

* Made a single MockEndpointSecurityAPI class to share everywhere

* Added most of SNTCompilerControllerTest

* Cleanup SNTCompilerControllerTest

* Started expanding Auth client test

* Finished up the Authorizer tests

* Move to using enum class for notify/auth instead of bool

* WIP for tamper resistance test. ASAN issues.

* Add OCMock patch to fix test issue on ARM Macs

* Changed patches directory name to external_patches

* Update WORKSPACE path

* Finished up Tamper Resistance tests

* Finished up Recorder tests.

* Move SNTExecutionControllerTest to ObjC++

* Initial work to port SNTExecutionControllerTest

* Finished porting SNTExecutionControllerTest.

* Added SNTExecutionControllerTest to list of unit tests

* Ported SNTEndpointSecurityDeviceManager.

* Test cleanup, use MockESAPI expectation helpers

* Verify SNTEndpointSecurityDeviceManager expectations differently

* Test cleanup, omit gTest param list where unused

* Log message cleanup

* Rename SNTApplicationTest to santad_test.mm

* Finished porting santad_test, formerly SNTApplicationTest

* Fix SNTEndpointSecurityDeviceManager issues

* Pulled in missed fixes. Updated tests.

* Renamed lowercase filenames to match rest of codebase

* Fix non-static dispatch_once_t, and noisy watching compiler log message

* WIP Started process of removing components no longer used

* WIP Continued process of removing components no longer used

* BUILD file cleanup. Proto warning. Removed unused global

* Rename SNTEventProvider to SNTEndpointSecurityEventHandler

* Rename SNTEndpointSecurityEventHandler protocol

* Remove EnableSysxCache option. Remove --quick flag used during dev.

* Ran testing/fix.sh

* Addmissing param to fix.sh that was omitting .mm files.

* clang-format

* Fix linter: find cmd missing .mm ext, git grep exclude patch files.

* Use MakeESProcess default params in tests

* Move variables to camelCase in objc classes

* More case changes

* Sanitize strings

* Change dispatch queue priorities and standardize daemon queue naming

* Exclude patch files in markdown check

* Ensure string log messages end with newline

* Fix BasicStringTest

* Disable clang-format in code producing different results in local/remote versions

* Moved to using date ranges in copyright notices as per current guidelines

* Update Source/common/SNTConfigurator.h

Suggestion adding whitespace in comment to fix clang-format mangling

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>

* Removed santa_panic macro used in one place

* Updated comment about ES cachability

* Pin oneTBB to specific commit

* Address outstanding WORKSPACE 'canonical reproducible form' messages

* Use string append instead of ostringstream due to benchmark results

* Remove use of freind classes in EnrichedTypes.h

* Added SNTKVOManager, removed observers from SNTConfigurator.

* Fixed SNTEndpointSecurityRecorderTest class name

* Reduce usage of the auto keyword

* Each SNTKVOManager instance now adds its own observer

* Replaced more auto keywords with real types.

* Remove leftover code coverage debugging from ci.yml

* Updated comment

* Memoize SNTFileInfo sha256. Reduce some cache sizes.

* Fix issue checking for translocated paths

* Use more performant NSURL creation method

* Fix lint issue

* Address PR feedback

* Use an array literal for kvo objects

* Fix some clang tidy and import issues

* Replace third party LRU cache with SantaCache for now

* Fix clang tidy issues

* Address PR feedback

* Fix comment typo

Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>

* Added todo for when we adopt macOS 13

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>
2022-09-22 10:18:41 -04:00
Matt W
c67364fe76 Protobuf support, maildir format logging (#731)
* Initial protobuf support, maildir logging

Fix build issues in the integration test

Deduped some test code

Formatting

Address feedback from draft PR

Removed legacy labels. Updated docs.

Add in metrics. Fix protobuf logging test.

* Now use the Any proto for the LogBatch wrapper

* Changes based on PR feedback

* Added gauge metrics for spool dir

* Formatting

* Add event time to proto

* Fix build issue after rebase

* Update BUILD rules

* Updated language around protobuf logging to mark as beta
2022-03-14 15:46:52 -04:00
Kent Ma
58cec5819a Add linter step with clang-format and buildifier. (#620)
Also lint our files accordingly
2021-10-01 16:51:06 -04:00
Kent Ma
6ba5831f2d Run buildifier (#619) 2021-10-01 15:18:33 -04:00
Kent Ma
a22e3ead83 Add regular execution integration tests (#618) 2021-10-01 15:07:56 -04:00