Use grep better to speed up crew remove of packages. (#10309)

* use grep to speed up crew remove

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* bump version

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* adjust logic

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Add back CREW_ESSENTIAL_FILES to handle files like libC.so.6

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* adjust CREW_ESSENTIAL_FILES logic

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* fix package_files_that_overlap

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* cleanup output of package_files_that_overlap

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* debug

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Back out CREW_ESSENTIAL_FILES change, and use Package.load_package('pkg.rb').get_deps_list to find dependent packages of CREW_ESSENTIAL_PACKAGES in commands/remove.rb, and also fix Package.load_package('pkg.rb').get_deps_list to enable using it from commands/remove.rb

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Adjust removal message in crew.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Adjust some punctuation and capitalization.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* fix pkg_file path detection

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Only use CREW_LOCAL_REPO_ROOT file if it exists.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* suggested changes

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Fix ESSENTIAL_PACKAGE dependency expansion.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Fix essential_deps logic.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Adjust remove logic to return quietly if in_upgrade, else exit 1, and also properly figure out essential_packages list.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Fix docopt so options like '-d' work.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Add suggested changes for testing, add testing file for remove.rb, also in testing.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* fixup

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* more testing

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Move recursive package function to package.rb

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Adjust function name.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* simplify

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* simplify

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Use File.mtime for @last_update_check as per suggestion by @Zopolis4.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Move essential package determination to crew:generate_compatible.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Move load_json and save_json to package_utils and just use symbols for the json array in remove and package_utils.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Unify json usage globally.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Fix one remove test.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* lint

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Update remove.rb test.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* cleanup

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Adjust remove test.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Try to use new Unit Tests...

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Fix installsh: git config --local commands cannot be run unless the git repo is already setup.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Move json functions to package_utils.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Fix docopt for real, add docopt unit test, have install.sh generate ruby gem filelists just like a regular buildsystems/ruby gem install.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Use CREW_META_PATH in install.sh.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Cleanup remove.rb tests.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Add 'crew list essential'.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Add unit test for list command.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Cleanup remove.rb test.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Cleanup wording.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Move print_deps_tree to lib/package.rb.

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* lint

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

* Return changes from #10317

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

---------

Signed-off-by: Satadru Pramanik <satadru@gmail.com>

lib/package_utils.rb

@@ -1,10 +1,27 @@
 require 'json'
 require_relative 'const'
+require_relative 'crewlog'
 
 class PackageUtils
+  def self.load_json
+    return JSON.load_file(File.join(CREW_CONFIG_PATH, 'device.json'), symbolize_names: true).transform_values! { |val| val.is_a?(String) ? val.to_sym : val }
+  end
+
+  def self.save_json(json_object)
+    crewlog 'Saving device.json...'
+    begin
+      File.write File.join(CREW_CONFIG_PATH, 'device.json.tmp'), JSON.pretty_generate(JSON.parse(json_object.to_json))
+    rescue StandardError
+      puts 'Error writing updated packages json file!'.lightred
+      abort
+    end
+
+    # Copy over original if the write to the tmp file succeeds.
+    FileUtils.cp("#{CREW_CONFIG_PATH}/device.json.tmp", File.join(CREW_CONFIG_PATH, 'device.json')) && FileUtils.rm("#{CREW_CONFIG_PATH}/device.json.tmp")
+  end
+
   def self.installed?(pkg_name)
-    device_json = JSON.load_file(File.join(CREW_CONFIG_PATH, 'device.json'))
-    return device_json['installed_packages'].any? { |elem| elem['name'] == pkg_name }
+    return load_json[:installed_packages].any? { |elem| elem[:name] == pkg_name }
   end
 
   def self.compatible?(pkg)