60 Commits

Author SHA1 Message Date
Tom Burgin
c711129ac9 s/NSDictionary/NSBundle/ (#1418) 2024-08-15 12:36:01 -04:00
Tom Burgin
c7766d5993 length and count check (#1413) 2024-08-09 17:17:13 -04:00
Tom Burgin
b1cf83a7e3 switch to CFBundleCopyInfoDictionaryInDirectory (#1411) 2024-08-08 16:30:33 -04:00
Matt W
74d8fe30d1 Creating transitive rules for rename events should fallback to destination path (#1299)
* Transitive rules should fallback to destination for RENAME events

* Add tests to exercise fallback for rename events
2024-02-28 17:09:07 -05:00
Nick Gregory
a2f8030482 Fuzz embedded plist reading (#990)
* fuzz embedded plist reading

* remove newline

* consolidate size checking

* brackets

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
2023-01-03 14:22:31 -05:00
Nick Gregory
ca81270bff Fix SNTFileInfo Fuzzing (#985)
* fix SNTFileInfo fd leak

* auto poweroff VM after fuzzing

* lint
2022-12-27 16:20:32 -05:00
Matt W
fcb49701b3 ES and Logging Interfaces Redesign (#888)
* Initial structure for ES wrappers, enriched types, logging

* Basic working ES and logging functionality

* Add in oneTBB and thread-safe-lru deps

* Added a bunch of enriched types

* Auto-mute self when establishing ES client

* Basic auth, tamper client. Syslog of all events. Basic compiler tracking.

* Update copyright header blobs, convert some tabs to spaces

* Auth result cache. Fix getting translocation path.

* Added remaining cache methods

* Add AuthResultCache to Recorder client. Cache now operates on es_file_t.

* Hooked up SNTPrefixTree

* Fix CompilerController for RENAME. Fix AllowList logging missing path.

* Block loading Santa kext

* Added device manager client

* Properly log DiskAppear events

* Fix build to adopt new adhoc build

* Handle clearing cache on UNMOUNT events

* Ignore other ES clients if configured

* Remove SNTAllowlistInfo. Rename AllowList to Allowlist. Minor cleanup.

* Recorder now logs asynchronously. Enricher now returns shared_ptrs.

* Added File writer. Added timestamps to BasicStream serializer.

* Skip calling stat in SNTFileInfo when path given by ES.

* Fix build issue

* Address draft PR feedback

* santactl integrated, XPC works, fix file writer bug

* Integrate syncservice. Start observing some config changes.

* Add metrics service wrapper

* Add metrics config observers and metrics interval reset.

* Start better dependency control. Add Null logger support.

* Added more deps

* Added more deps

* Fix issue where metric service wasn't starting

* Add missing variant include

* Fix missing parent proc name

* Added googletest and new unit test macro

* Started expanding AuthResultCacheTest

* Properly mock EndpointSecurityAPI

* Finished AuthResultCacheTest

* bazelrc now builds all C++ as C++17. Added LoggerTest.

* Add FileTest. Abstract some File constants to Logger.

* Added Empty serializer test

* Started work on BasicStringTest. Fixed some BasicString serialization bugs.

* Added Unlink BasicString serialization test

* Added some more tests. Commonized some test code

* Finished BasicStringTest. Converted to XCTest.

* Standardize esapi variable naming

* Bubble up gTest expect failures to XCTest failures

* AuthResultCacheTest now uses XCTest. Added common TestUtils.h

* EmptyTest now uses XCTest.

* FileTest now uses XCTest

* LoggerTest now uses XCTest. Removed santa_unit_gtest bazel macro.

* Added ClientTest

* Add basic Enricher tests

* Add MessageTest. Make more TestUtils.

* Rename metrics to Metrics

* Add MetricsTest.

* Apply template pattern to Serializer

* Add SNTDecisionCacheTest.

* Add SNTCachedDecisionTest.

* Testing with coveralls debug mode

* Allow manual CI runs

* Remove unused property

* Started work on SNTEndpointSecurityClientTest.

* WIP SNTEndpointSecurityClientTest, fix test run issue

* Added more base ES client tests

* Add more base ES client tests

* Base ES client tests done. Added serializer utils/tests. Expanded basic string tests.

* Add utils test to test suite

* Add copy ctor. Add test output to bazel coverage.

* Single thread bazel coverage

* Updaload coverage file

* Updaload coverage file

* Old gen cov test

* Restructure message handlers to enable better testability

* Added enable tests for all ES clients

* Made a single MockEndpointSecurityAPI class to share everywhere

* Added most of SNTCompilerControllerTest

* Cleanup SNTCompilerControllerTest

* Started expanding Auth client test

* Finished up the Authorizer tests

* Move to using enum class for notify/auth instead of bool

* WIP for tamper resistance test. ASAN issues.

* Add OCMock patch to fix test issue on ARM Macs

* Changed patches directory name to external_patches

* Update WORKSPACE path

* Finished up Tamper Resistance tests

* Finished up Recorder tests.

* Move SNTExecutionControllerTest to ObjC++

* Initial work to port SNTExecutionControllerTest

* Finished porting SNTExecutionControllerTest.

* Added SNTExecutionControllerTest to list of unit tests

* Ported SNTEndpointSecurityDeviceManager.

* Test cleanup, use MockESAPI expectation helpers

* Verify SNTEndpointSecurityDeviceManager expectations differently

* Test cleanup, omit gTest param list where unused

* Log message cleanup

* Rename SNTApplicationTest to santad_test.mm

* Finished porting santad_test, formerly SNTApplicationTest

* Fix SNTEndpointSecurityDeviceManager issues

* Pulled in missed fixes. Updated tests.

* Renamed lowercase filenames to match rest of codebase

* Fix non-static dispatch_once_t, and noisy watching compiler log message

* WIP Started process of removing components no longer used

* WIP Continued process of removing components no longer used

* BUILD file cleanup. Proto warning. Removed unused global

* Rename SNTEventProvider to SNTEndpointSecurityEventHandler

* Rename SNTEndpointSecurityEventHandler protocol

* Remove EnableSysxCache option. Remove --quick flag used during dev.

* Ran testing/fix.sh

* Addmissing param to fix.sh that was omitting .mm files.

* clang-format

* Fix linter: find cmd missing .mm ext, git grep exclude patch files.

* Use MakeESProcess default params in tests

* Move variables to camelCase in objc classes

* More case changes

* Sanitize strings

* Change dispatch queue priorities and standardize daemon queue naming

* Exclude patch files in markdown check

* Ensure string log messages end with newline

* Fix BasicStringTest

* Disable clang-format in code producing different results in local/remote versions

* Moved to using date ranges in copyright notices as per current guidelines

* Update Source/common/SNTConfigurator.h

Suggestion adding whitespace in comment to fix clang-format mangling

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>

* Removed santa_panic macro used in one place

* Updated comment about ES cachability

* Pin oneTBB to specific commit

* Address outstanding WORKSPACE 'canonical reproducible form' messages

* Use string append instead of ostringstream due to benchmark results

* Remove use of freind classes in EnrichedTypes.h

* Added SNTKVOManager, removed observers from SNTConfigurator.

* Fixed SNTEndpointSecurityRecorderTest class name

* Reduce usage of the auto keyword

* Each SNTKVOManager instance now adds its own observer

* Replaced more auto keywords with real types.

* Remove leftover code coverage debugging from ci.yml

* Updated comment

* Memoize SNTFileInfo sha256. Reduce some cache sizes.

* Fix issue checking for translocated paths

* Use more performant NSURL creation method

* Fix lint issue

* Address PR feedback

* Use an array literal for kvo objects

* Fix some clang tidy and import issues

* Replace third party LRU cache with SantaCache for now

* Fix clang tidy issues

* Address PR feedback

* Fix comment typo

Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>

* Added todo for when we adopt macOS 13

Co-authored-by: Russell Hancox <russellhancox@users.noreply.github.com>
Co-authored-by: Pete Markowsky <pmarkowsky@users.noreply.github.com>
2022-09-22 10:18:41 -04:00
Kent Ma
d2b6c2b6c2 Update .clang-format and apply to existing files. (#558) 2021-06-15 09:47:31 -04:00
Russell Hancox
78643d3c49 fileinfo: Don't use non-bundle dirs as possible ancestors (#537) 2021-02-01 11:09:32 -05:00
Russell Hancox
4f65965277 santactl/fileinfo: Fix fileinfo tests on BigSur + multiarch plists (#523)
The fileinfo tests didn't work on BigSur because of some path and binary changes.

Also, the embeddedPlist method didn't work on fat binaries, of which there are now
many, because of M1 machines. I think we didn't notice this before because we pull
the embedded plist from the first arch listed in the headers dict which generally
seemed to pick x86_64 first but with the arm64/arm64e option being added
that now appears first.

Also fixed some errors handling 32-bit segment/sections and added a test for this.
2021-01-07 19:46:48 -05:00
Russell Hancox
bf6f78df09 common: Eliminate VLA usage in SNTFileInfo (#435)
VLAs complicate static analysis and bloat stack size. Replace VLA allocation with calls to malloc and free
2020-02-11 10:55:57 -05:00
Russell Hancox
dc692c8256 Project: Move tests with the code they're testing (#343)
Add helper to make declaring unit tests easier
Add unit_tests test_suite containing all unit tests
Fix reload rule
Update to workspace-relative header locations that were missed before
2019-02-06 15:09:09 -05:00
Tom Burgin
798b0fab15 fileinfo: whitelist bundle extensions when performing an ancestor search (#257)
* fileinfo: whitelist bundle extensions when performing an ancestor search

* fix tests

* conform to russell's every whim
2018-05-07 13:57:58 -04:00
Russell Hancox
e8630132d7 Project: make building on case-sensitive volumes work (#255) 2018-04-27 12:58:54 -04:00
Tom Burgin
5ee6531627 santad / santactl: validate all architectures within universal binaries (#249) 2018-04-24 16:11:49 -04:00
nguyen-phillip
5216f0989c santactl: Recursive fileinfo command (#191)
* temporarily gutted SNTCommandFileInfo. Added SNTCommand base class for all
of the SNTCommand* classes to inherit from.  Changed commands so that they
are consistently instantiated before being run, with a common init method.

* Put most of SNTCommandFileInfo functionality back in

* follow symlinks

* added -r and --recursive flags and updated help text

* moved humanReadableFileType to SNTFileInfo

* added back JSON output

* Fixed bundle info. Grab directory color from ENV variable.

* fixed indentation, moved stuff around

* Added SNTCommandFileInfo * back as parameter to property getters so that rule getter
doesn't have to be a special case any more.

* fixed code review issues

* added SNTCommand.h and SNTCommand.m to project

* added SNTCommand.m to build phases

* removed trailing spaces

* fixed tests for SNTCommandFileInfo and added a few more

* fix end-of-line comment spacing to conform to style guide

* Use NSBundle instead of NSWorkspace to determine if path is a bundle.

* added autorelease pool inside recursive search loop to fix bug where file listing
would abruptly stop after so many files with mach header related keys.

* removed directory headers. don't separate entries with newline when printing single key. format output based on max key length.

* an attempt at speeding things up.  also halfway fixed broken cert-index key.

* speedups via caching MOLCodeSignChecker & not using NSMutableString append*

* fix json ouput with cert-index, single key output, & cache SHA values

* reverted back to NSMutableString for building up output, since it seems slightly better
or at least no worse than using an NSMutableArray

* Don't print empty JSON objects

* fixed non-thread-safe JSON commas

* made the print dispatch group a property so it doesn't have to be passed around

* Fixed certIndex indexing bug & better error checking when parsing --cert-index argument

* prevent unsigned int overflow

* fixed logic tests broken by objc_setAssociatedObject with nil SNTFileInfo argument

* send error output to the serial print queue

* NSBundle bundleWithPath: returns an object even for non-bundle directories, so need to also check that there's a valid bundle identifier.

* Added TODO comment and fixed formatting issues

* added cached codeSignChecker property to SNTFileInfo

* rewrote SNTFileInfo's codesignChecker method to include an error reference parameter & removed @synchronized

* Removed caching of SHA values from SNTFileInfo

* use property getter/setter to access codesignCheckerError

* Change nil NSError ** arguments to NULL

* Don't try to create a new codesignChecker if there was previously an error

* Fix NSDirectoryEnumerator memory usage & don't retain self in rule getter.

The NSStrings grabbed from the directory enumerator needed a chance to be freed.

* fixed colon alignment
2017-08-18 09:56:37 -04:00
Russell Hancox
616fd9570f santa-driver: Split cache for root/non-root volume
Split the kernel-land cache into 2 separate caches, one for the root
volume and one for secondary volumes. When an unmount happens, clear
the non-root cache to ensure no overlap with filesystem IDs.
2017-08-16 12:07:44 -04:00
Tom Burgin
363826502f santabs: de-dupe generated events before upload (#177)
* santabs: de-dupe generated events before upload and remove locks

* review updates

* error updates
2017-06-22 17:46:04 -04:00
Tom Burgin
d3b3d722b4 santabs: use the ancestor bundle when searching for binaries (#175)
* santabs: use the ancestor bundle when searching for binaries

* review updates

* bundle tests
2017-06-09 17:04:23 -04:00
Tom Burgin
96150a9668 Bundle Events (#145)
* santabs: Create Santa Bundle Service

* common: SNTXPCConnection add initClientWithServiceName:

* santad: add logic for blocked bundles

* SantaGUI: add ui elements and xpc connections to / from santabs

* santactl/sync: add api features for syncing bundle events

* santactl/bundleinfo: add bundleinfo command for debug builds

* common: prefer bundle hash over file hash for event urls

* common: remove syncBackoff property - this is now handled in santactl sync

* common: add properties to support the bundle event api

* common: find a bundle from a nested binary

* review updates

* sane bundle hash time outs

* post rebase updates

* post review updates
2017-04-07 15:31:56 -04:00
Russell Hancox
a5fa6c7aef santactl/fileinfo: Recognize bundle/plugin mach-o files. 2016-08-22 14:05:22 -04:00
Russell Hancox
7600506d6d santad: Include client mode in execution logs. 2016-08-18 14:44:40 -04:00
Russell Hancox
df8e41925f SNTFileInfo: Check NSURLQuarantinePropertiesKey is usable 2016-07-13 17:29:53 -04:00
Russell Hancox
687ecc7097 santad: Close more file descriptors on exec 2016-07-11 16:23:38 -04:00
George Kola
7a3a98c27a Correctly use pread
pread can return less than the chunk size (e.g. signal caught in the
middle) and hence we need to handle it. This change also cleans up the
hash function and makes it more performant.
2016-06-29 11:21:56 -07:00
George Kola
d388e99c0e Cache method call
Minor optimization. Cache objc method call in local variable to avoid a
second call
2016-06-28 21:26:35 -07:00
Russell Hancox
2baea9a6b4 Project: Xcode recommended updates. 2016-06-28 17:34:58 -04:00
Russell Hancox
9058192ffe santad: Use memcpy instead of strncpy where appropriate 2016-06-28 16:23:06 -04:00
Russell Hancox
8479730c95 SNTFileInfo: Catch potential NULL-pointer deref in isScript and isXARArchive. 2016-06-28 14:54:21 -04:00
Russell Hancox
7102e2df4c SNTFileInfo: More speed-ups in hashing, use RDAHEAD, don't use NOCACHE, catch EINTR. 2016-06-28 14:52:28 -04:00
Russell Hancox
c560405a46 SNTFileInfo: Speed up hashing - increase chunksize, read directly, use fcntl
- Use fcntl to disable cache and issue an advisory read
- Increase default chunk size from 4KB to 256KB
- Use pread to read from file descriptor, rather than make NSData objects

This is ~15% faster.
2016-06-27 17:38:41 -04:00
Russell Hancox
aefd85455e Project: s/OS X/macOS/g 2016-06-16 17:31:40 -04:00
Russell Hancox
88e3a606a0 SNTFileInfo: Use CFBundleDisplayName if available 2016-04-26 17:34:29 -04:00
Russell Hancox
43434fd445 santactl/fileinfo: Don't crash on <512b files 2016-04-08 16:20:49 -04:00
Russell Hancox
e0a46be1b7 santactl/fileinfo: When resolving path, store bundle ref if possible. 2016-03-14 12:55:20 -04:00
Russell Hancox
fd82c67b56 santactl/fileinfo: Add disk image file type 2016-03-14 12:55:20 -04:00
Russell Hancox
f0a83b6f19 santactl/fileinfo: Add simultaneous hashing. 2016-03-14 12:52:25 -04:00
Russell Hancox
0e00237e44 Project: Add clang-format file, apply most of the fixes it suggested 2016-03-10 15:53:06 -05:00
Russell Hancox
b6487000a3 SNTFileInfo: Use NSBundle to find executable path in bundles.
Fixes #37
2016-03-10 12:19:52 -05:00
Russell Hancox
d507e79505 santad: Fix quarantine data collection.
This previously didn't work for root (santactl fileinfo was fine)
because quarantine data is per-user.
2016-03-07 12:30:36 -05:00
Russell Hancox
d8a8aba0ea SNTFileInfo: Move machoType method to binaryinfo command, add XAR archive detection. 2015-12-14 17:25:32 -05:00
Russell Hancox
36189e9122 santad: Update SNTFileInfo to always get strings from bundle Info.plist data.
Also perform a one-time update of any events created before this change.
2015-12-04 13:09:56 -05:00
Russell Hancox
209eaff3c6 SNTFileInfo: Embed SHA hashing loop in an autoreleasepool to avoid temporary RAM spikes 2015-10-31 13:45:47 -04:00
Russell Hancox
77c46b5c43 SNTFileInfo: switch from NSData to NSFileHandle.
This seems to work much better than NSData with either mapped (SIGBUS when file is deleted) or uncached (ballooning memory use) reading.
2015-10-29 16:17:12 -04:00
Russell Hancox
33a7b38c6a SNTFileInfo: check for NULL ptrs when parsing for embedded plist 2015-10-27 18:35:11 -04:00
Russell Hancox
2a7c0bd58c SNTFileInfo: Go back to using mmap, uncached read balloons memory use 2015-10-27 18:08:16 -04:00
Russell Hancox
63f65c51c3 SNTFileInfo: Use NSURL method for getting quarantine data, don't try to use <10.10 2015-10-15 12:14:53 -04:00
Russell Hancox
bc51c9f25b SNTFileInfo: Add com.apple.quarantine data accessors for downloaded files. 2015-10-14 20:11:32 -04:00
Russell Hancox
c412e8b9a7 SNTFileInfo: Fix embedded plist parsing, extract into separate method 2015-10-14 20:07:50 -04:00
Russell Hancox
13aa889633 SNTFileInfo: Add fileSize method, use it in SNTEventLog 2015-10-08 17:57:02 -04:00