Setting up a custom namespace

Custom namespaces serve as logical environments to package and isolate your data foundation and analytical data product modules. Running inside a dedicated custom namespace lets you manage configuration and deployment options independently, providing isolation for your environment. A key benefit of this separation is that you can seamlessly pull future updates and enhancements from the cortex namespace in Google Cloud Cortex Framework without the risk of overriding or corrupting your custom modules. We highly recommend creating your custom modules within a dedicated namespace.

Custom namespace folder structure

The Cortex Framework namespace folders are used to package and isolate custom module artifacts. The folder structure of a custom namespace is defined in the table:

Directory path	Purpose and description
`config/`	Cortex deployment configuration (Required) Creating new namespaces requires their setup within `config.yaml`.
`src/data_modules/custom_namespace/data_foundation/data_foundation_module_type`	Data foundation modules Raw-to-foundation datasets transformations. Each data foundation is separated in a subdirectory and consists of: `annotations/`: Column and field-level descriptions. `table_settings.default.yaml`: Table settings configuration
`src/data_modules/custom_namespace/data_product/data_product_module_type`	Data product definitions Contains business logic and analytical models. Each product is separated in a subdirectory and consists of: `Readme`: Documentation of the module `manifest.yaml`: Declares the builder type, dependencies, and configuration. `table_settings.default.yaml`: Table settings configuration `annotations/`: Column and field-level descriptions. `definitions/`: Dataform SQLX or JS source files.
`src/data_modules/custom_namespace/includes/`	JavaScript helpers Any JavaScript files placed here are automatically packaged and made available to your Dataform models during compilation under the namespace path (for example, `includes/custom_namespace/`).
`src/data_modules/custom_namespace/common/`	Namespace shared tools Namespace-scoped custom builders, deployers and other tools.

Custom namespace configuration

A new namespace: custom_namespace can be defined by extending the corresponding section of the config.yaml file, creating a new directory under src/data_modules/ and adding the new module code assets.

To make the Google Cloud Cortex Framework compiler aware of your custom namespace and modules, you must register them in your global configuration file (e.g., config/config.yaml).

Step A: Register the namespace

Under the data.namespaces block, add your namespace metadata:

data:
  namespaces:
    - name: cortex
      path: ../src/data_modules/cortex
    - name:  custom_namespace    # <-- Name of custom namespace
      path:  ../src/data_modules/custom_namespace  # <-- Points to subdirectory under 'src/data_modules/'

Step B: Configure custom modules

Register your custom modules under data.modules.foundation or data.modules.product. Use the dot-separated format custom_namespace.module_type for the type field:

data:
  modules:
    foundations:
      - moduleId:  custom_namespace_data_foundation_module_type
        modulePath: custom_namespace.sap   # Format: <namespace>.<module_type>
        dataSourceId: sap_raw_s4
        dataTargetId: data_foundation_sap_custom_namespace
        moduleSettings:
          sapVersion: s4
          mandt: "100"