[FRONTEND] Enable triton to support register thirdparty backend at runtime (#1643)

This PR intends to provide a mechanism to support a third-party backend
at runtime to generate the backend-specific code.

The mechanism provided a common class to abstract the third-party
backend logic and two essential functions to register and get the
third-party backend at runtime.

- `BaseBackend`: A common class to abstract the third-party backend
logic
- `register_backend`: Register a third-party backend with a given device
type
- `get_backend`: Get the third-party backend with a given device type

Generally, a third-party backend must inherit from `BaseBackend` and
implement all the member functions according to the backend
characteristics. As long as the backend implementation is ready, the
third-party backend can invoke `register_backend` to register it under a
given device. During the kernel compilation and execution, the mechanism
will get the registered backend to generate the kernel and launcher code
for a given device.

This PR added a dummy backend to simulate a third-party backend and
demonstrate the usage.

-
[test_device_backend.py](https://github.com/openai/triton/pull/1643/files#diff-bbe4d50624f2d11bf17c878a1ed4d422918c124c182cf9357b993240c385bea1):
To define a third-party backend and register the backend
-
[ExtensionBackend](https://github.com/openai/triton/pull/1643/files#diff-bbe4d50624f2d11bf17c878a1ed4d422918c124c182cf9357b993240c385bea1R123):
Inherit from the `BaseBackend` and implement some specific logic like
[filter out some compile
stages](https://github.com/openai/triton/pull/1643/files#diff-bbe4d50624f2d11bf17c878a1ed4d422918c124c182cf9357b993240c385bea1R129-R135)
- [Register the `ExtensionBackend` for
`CPU`](https://github.com/openai/triton/pull/1643/files#diff-bbe4d50624f2d11bf17c878a1ed4d422918c124c182cf9357b993240c385bea1R279)
  
-
[extension_backend.c](https://github.com/openai/triton/pull/1643/files#diff-169c1d08b3a0a7b343cfa3258fbc32b47e0f6c46305a112652fa1bdaaec89d29):
To provide the utility function to load kernel binary and get the
backend properties.

python/setup.py

@@ -31,6 +31,17 @@ def get_build_type():
         # TODO: change to release when stable enough
         return "TritonRelBuildWithAsserts"
 
+
+def get_codegen_backends():
+    backends = []
+    env_prefix = "TRITON_CODEGEN_"
+    for name, _ in os.environ.items():
+        if name.startswith(env_prefix) and check_env_flag(name):
+            assert name.count(env_prefix) <= 1
+            backends.append(name.replace(env_prefix, '').lower())
+    return backends
+
+
 # --- third party packages -----
 
 
@@ -210,6 +221,11 @@ class CMakeBuild(build_ext):
         cfg = get_build_type()
         build_args = ["--config", cfg]
 
+        codegen_backends = get_codegen_backends()
+        if len(codegen_backends) > 0:
+            all_codegen_backends = ';'.join(codegen_backends)
+            cmake_args += ["-DTRITON_CODEGEN_BACKENDS=" + all_codegen_backends]
+
         if platform.system() == "Windows":
             cmake_args += [f"-DCMAKE_RUNTIME_OUTPUT_DIRECTORY_{cfg.upper()}={extdir}"]
             if sys.maxsize > 2**32:
@@ -256,9 +272,7 @@ setup(
         "triton/ops/blocksparse",
         "triton/runtime",
         "triton/runtime/backends",
-        "triton/third_party/cuda/bin",
-        "triton/third_party/cuda/include",
-        "triton/third_party/cuda/lib",
+        "triton/third_party",
         "triton/tools",
     ],
     install_requires=[