High Availability Deployment for the Filigran OpenCTI


In a recent scenario, I found myself assisting a friend who had encountered a connectivity challenge with an old version of MS SQL database.
The unique aspect of this situation was the authentication method employed by the database, which relied on Windows Authentication as the default protocol.
This became a particularly intricate issue for my friend, who, as a new employee, found himself amidst a predominantly Windows-oriented environment within the company.
Complicating matters further, my friend, a dedicated Mac user, faced uncertainty in identifying the appropriate person within the organization to seek assistance from.
In light of these circumstances, I stepped in to provide guidance and support, ensuring a successful connection to the MS SQL database on his Mac system.

## Situation

The company my friend works for has its own VPN, which he had already installed on his Mac.
The company policy doesn't allow employees to connect to the database by using a username and password, but only by using Windows Authentication.

He had also installed the Azure Data Studio, because there is no SQL Server Management Studio (SSMS) for Mac.
However, the Azure Data Studio, with Windows Authentication, keeps asking for kinit (Kerberos) credentials, which he doesn't have.
And yes, running `kinit` doesn't help because it is not yet right configured on his Mac.
![image](/images/post/post-4-mssql-from-mac/azure-error.webp)

Trying with DBeaver, it also doesn't work.
While other Windows users can use the `mssql-jdbc_auth` library to connect to the database, the DLLs are not for MacOS.
![image](/images/post/post-4-mssql-from-mac/dbeaver-driver-error.webp)

## Solution
Actually, the solution is quite simple, but it took me a while to figure it out.
The right authentication method is to use the Integrated Windows Auth (NTLM).

I couldn't find out how to use the Integrated Windows Auth (NTLM) with Azure Data Studio, but I found out how to use it with DBeaver and IDE like PyCharm.

### DBeaver
Using the NTLM and the JDBC driver from Microsoft, we can connect to the database with our domain username and password.
![image](/images/post/post-4-mssql-from-mac/dbeaver-ntlm.webp)

### PyCharm
To setup the IntelliJ IDE like PyCharm to connect to the database, we need to do the following:
1. Using `jtds` Driver that does support Windows Authentication
2. Using `Domain Authentication` authentication method
3. Visiting the Advance tab and enable the `USENTLMV2` attribute.

Ref. https://www.jetbrains.com/help/pycharm/db-tutorial-connecting-to-ms-sql-server.html#connect-by-using-windows-domain-authentication-macos-and-linux

![image](/images/post/post-4-mssql-from-mac/pycharm-ntlm.webp)
![image](/images/post/post-4-mssql-from-mac/pycharm-ntlm-2.webp)

## Kerberos and NTLM

Firstly, here is a good article about Kerberos and NTLM:

> Both Kerberos and NTLM are important authentication protocols used in Windows environments. However, Kerberos is more secure, scalable, and compatible with modern systems, while NTLM is more straightforward to configure and manage and works well with older systems.
> https://www.geeksforgeeks.org/difference-between-kerberos-and-ntlm/

It means that in another scenario, we may encounter the opposite situation, where the database supports Kerberos but not NTLM.
In that case, we need to use Kerberos to connect to the database.

### Kerberos
There is a quite good article about using Kerberos on MacOS to connect to MS SQL Server with Windows Authentication in Azure Data Studio below.
* https://medium.com/@chriskuchar/connecting-to-ms-sql-server-from-a-mac-with-windows-authentication-44375ec1953e

The only thing I want to add is where we can find the domain name and KDC server to fill in the `krb5.conf` file.
```editorconfig
# File /etc/krb5.conf 

[libdefaults]
default_realm = DOMAIN_IN_CAPPITAL_LETTERS
 
[realms]
DOMAIN_IN_CAPPITAL_LETTERS = {
kdc = kdc.address.in.domain
}
```

First, the simplest way is to use a Windows machine, that already connected to VPNs if your organization has one,
to run the `cmd` command below to get the domain name and KDC server.
```bash
# I'm not sure if `dig` can list all the domain names and KDC servers, but `nslookup` can.
# If you happen to know how to do it with `dig`, please let me know.
# At least the command `scutil --dns` in Mac will return the Domain Name Servers (DNS) of the VPN.
nslookup -type=any _kerberos._tcp 
```
A list of domain names and KDC servers will be returned, that we can use any of them to fill in the `krb5.conf` file.
![image](/images/post/post-4-mssql-from-mac/kdc-domain.webp)
```editorconfig
# File /etc/krb5.conf 

[libdefaults]
default_realm = DOMAIN.COM
 
[realms]
DOMAIN.COM = {
kdc = kdc-1.domain.com
}
```

<Notice type="info">
  **What if we don't have a real Windows machine?**

  The first option is to setup a Windows Virtual Machine on our Mac. Software that can help us to temporarily standup a Windows VM are Parallels, VMWare, etc.
  You don't need to install VPN on your virtual machine, if it is already enabled on your Mac. In default, your VM uses the same network as your Mac.
  
  And certainly, if you don't want many steps above, just use the Windows VM directly for your work. To connect to the organization's database as a normal Windows user.
  ![image](/images/post/post-4-mssql-from-mac/MSSQL-virtual-machine-windows.drawio.webp)
  ![image](/images/post/post-4-mssql-from-mac/window-vm.webp)
</Notice>

Now, after the `krb5.conf` file is configured, we can get a valid Kerberos ticket by running the command below.
```bash
kinit USERNAME@DEFAULT_REALM
# and enter your password when prompted
```
After it is done without any error, we can see our ticket by running the command below.
```bash
klist
```

With a valid Kerberos ticket, we can connect to the database that supports Kerberos with Azure Data Studio or any client.

## References
* https://stackoverflow.com/questions/8616818/integrated-windows-auth-ntlm-on-a-mac-using-google-chrome-or-safari
* https://learn.rdr-it.io/Le%C3%A7on/authentification-ntlm-et-kerberos/


From MacOS to MS SQL Server with Windows Authentication


The journey began when I sought a straightforward note-taking app for my everyday needs. Several months ago, I experimented with various online note-taking platforms for work. However, my boss, upon noticing my utilization of these apps, emphatically insisted that I refrain from entering any information into online note-taking tools due to security concerns that were not approved by the company.

While my employer provided a robust tool in the form of Confluence—a powerful documentation platform—it posed a challenge. Confluence, designed for long-term documentation, proved cumbersome for quick note-taking. Despite attempting to adapt it for personal note-taking, its lack of convenience and speed became apparent. Creating a quick note in Confluence necessitated opening a new (Chrome) tab, navigating to my personal space, creating a new page, and then initiating the writing process—only deciding later whether to publish or keep it as a draft. However, drafts were not easily visible for later review.

Similar challenges were encountered with other documentation systems such as Google Docs, Notion, and Evernote. What I sought was simplicity and speed—a tool that required minimal deliberation. I envisioned a process where I could open the tool and immediately start typing, devoid of unnecessary steps and decisions. The aim was to easily review and decide whether to retain or delete these quick notes later.

Faced with this need for a secure and convenient note-taking solution, I explored various options, including online note-taking apps, Chrome extensions, and applications. However, none met my criteria, as they were either too complex, had poor user experiences, or lacked the necessary security measures.


## Problems

1.  Documenting tools are powerful but not suitable for quick note-taking.
2.  Trust issue. Online taking notes apps are not secure enough (we are ).
3.  Current Chrome Note-taking extensions don't offer a good user experience and don't guarantee security.

## Solution

### Chrome Extension
What I need in this use case is a simple note-taking app that is secure and convenient to use, that seems to be a Chrome extension.
There are reasons for this choice:
1. Chrome extensions are easy to install and use.
2. Chrome extensions are secure because they are sandboxed ([localStorage](https://developer.chrome.com/docs/extensions/reference/storage/) that only this extension can access).
3. Data is saved in a Local Storage, which is fast to access. 
   - It is not shared with any third party.
   - Can be used offline, which is convenient for me to use on the go.
   - The only downside is that it is not synced across devices and is limited in size, but I don't need it in this use case. Maybe in the future, I will consider to add the option to sync data across devices.
4. It provides a good user experience with `sidePanel`, that users don't need to open new or switch to another tab to take notes.

### Svelte or not Svelte?

Prior to developing Ultra Notes, I had previously gained experience creating a couple of straightforward Firefox add-ons and Chrome extensions using vanilla JavaScript.

Here are the projects I worked on:

*   [Firefox add-on Count Word Professional](https://www.youtube.com/watch?v=8yzfDwRCtvg) in 2013, a decade ago, which has since been deprecated.
*   [Chrome extension X Word Replacer](https://chromewebstore.google.com/detail/x-word-replacer/efcnkaeocmkpckkenjinmdgkbjhajoca) in 2018, five years prior.

However, for a more intricate user experience, specifically incorporating a Directory Tree with multiple levels of folders and reused components,
I opted to use a frontend framework this time.
Among the available options like React and Svelte, I chose Svelte due to its lightweight nature,
a crucial property of Chrome Extension, to respond fast with limited resource in the browser.
Additionally, this decision aligns with my desire to learn Svelte, making this project an ideal opportunity for hands-on learning.
After all, the best way to learn is by doing, isn't it?
Instead of creating the project from scratch, I used the [SvelteKit](https://kit.svelte.dev/) and instruction from [Writing Chrome Extensions using Svelte Kit and Manifest V3](https://dev.to/michmich112/writing-chrome-extensions-using-svelte-kit-and-manifest-v3-mkd).

There is another good writing from the same author, that I think it is good to read before deciding to use Svelte or not in your Chrome extension project, [Why I chose Svelte for my Chrome Extension](https://dev.to/michmich112/why-i-chose-svelte-for-my-chrome-extension-3j4j).

<Notice type="info">
**Cons**

Your IDE will scream bloody murder if you try to use the chrome api's (as its not expecting those to be present).
You will still have to write custom JS to have a deeper interaction with the current tab/webpage (outside of what is offered through the chrome api).

**Caveat**

You are unable to use Svelte on any injected files. This means that you cannot use it to control the DOM on the user's window, only the DOM that is provided for the popups/action/alerts from the Chrome extension context. Thus if your app has to interact or retrieve information from the webpage without the extension popup being open, you'll have to write custom JS to manipulate the DOM, maybe even use JQuery to help you in doing so. This could have been possible previously with Chrome's Manifest V2 but with the deprecation of V2 and stable rollout of V3, many more restrictions have been put in place.
</Notice>

## Conclusion
I developed the extension to cater to my specific needs, with the hope that it proves equally useful for you.
As of the time of writing this post, the Ultra Notes extension stands at version 1.0.8 and is available on the [Chrome Web Store](https://chrome.google.com/webstore/detail/ngfhnakbmjknpcepkeiilllocgbdfehi/) for users' convenience.

The Ultra Notes extension seamlessly utilizes the Chrome built-in feature known as the [sidePanel](https://developer.chrome.com/docs/extensions/reference/sidePanel/), providing a satisfactory user experience.
Looking ahead, to further enhance the user interface and experience in future versions, I plan to incorporate [Content scripts](https://developer.chrome.com/docs/extensions/mv3/content_scripts/). 
For instance, envision a floating button or UI on any web page, surpassing the current Icon button on the top bar.
Achieving this may involve exploring ways to integrate Svelte into the content script or considering the use of another framework, such as React. 
The goal is to continually improve and expand the functionality of the Ultra Notes extension to meet the diverse needs of users.

One possible solution at the moment for bundling needed JS files for the content script is to use [Rollup](https://rollupjs.org/guide/en/), which is the bundler used by SvelteKit, or [Webpack](https://webpack.js.org/).

## References

The extension is hosted on Github at [Chrome Extension Ultra Notes](https://github.com/lhviet/chrome-extension-ultra-notes/), that is currently private, but will be made public/open source soon.
For the community trust, that your data is not sent to any third party, you can check the source code of the extension.

* https://github.com/lhviet/chrome-extension-ultra-notes/
* https://dev.to/michmich112/writing-chrome-extensions-using-svelte-kit-and-manifest-v3-mkd
* https://dev.to/michmich112/why-i-chose-svelte-for-my-chrome-extension-3j4j
* https://rollupjs.org/
* https://webpack.js.org/


Ultra Notes, a simple note taking Chrome extension built with Svelte


Officially, at this moment, we cannot simply add CesiumJS as a normal module into packages.json and use it directly as other libraries such as `lodash.` Instead, to use CesiumJS in your React project, there are some configurations needed.

In this post, I will review the current instructions and supplement with my practical experience to make CesiumJS works best in a Typescript-based ReactJS project.

### 1. Introduction

1. There is a bridge-library introduced by [_Hiroki Inoue_](mailto:hrknoue@gmail.com)  to lets you use CesiumJS easier, called [resium](https://medium.com/r/?url=https%3A%2F%2Fgithub.com%2Fdarwin-education%2Fresium), that you can find the post at
2. Indeed, using the wrapped-library is the easiest way. However, I prefer to have the full control on Cesium, therefore I went to the next solution, that I found by Google. A post from 2017, a little bit outdated.
   The post is useful and goes closer to the truth. However, it is not detailed enough and not a clean solution. Anyway, the solution introduced does work.
3. The best solution I found is officially provided by Cesium at [https://cesium.com/docs/tutorials/cesium-and-webpack/](https://medium.com/r/?url=https%3A%2F%2Fcesium.com%2Fdocs%2Ftutorials%2Fcesium-and-webpack%2F)

It has a cleaner setup and works well. And to make it easier for the configuration, I would like to add some additional steps in the next section, **Setup**.

### 2. Setup

Following the guide step by step at [https://cesium.com/docs/tutorials/cesium-and-webpack/](https://medium.com/r/?url=https%3A%2F%2Fcesium.com%2Fdocs%2Ftutorials%2Fcesium-and-webpack%2F)

1.  Along with packages added in the guide, I also add the package **@types/cesium** to use with Typescript (for function recommendation…)
```
// package.json
{
  "dependencies": {
    ...
    "cesium": "^1.62.0"
  },
  "devDependencies": {
    ...
    "@types/cesium": "^1.59.2",
    "copy-webpack-plugin": "^5.0.4"
  }
}
```

2. My project webpack structure is as below
```
├── config
│   ├── path.js                     // Default path
│   ├── webpack.config.common.js    // Common webpack configurations
│   ├── webpack.config.dev.js       // Inherit from common webpack
│   └── webpack.config.prod.js      // Inherit from common webpack
├── src
├── dist                   // Compiled .js files for deployment
├── node_modules           // installed packages
├── .env                   // dotenv configuration
├── .eslintrc              // Define eslint rules
├── .gitignore             // standard git ignore file
├── package.json           // scripts & packages
├── README.md              
└── yarn.lock
```

**path.js** defines the paths to use in webpack config.
```javascript
// The path to the CesiumJS source code
const cesiumSource = 'node_modules/cesium/Source';
const cesiumWorkers = '../Build/Cesium/Workers';
```

However, to make it easier in the config, I explicitly pre-define all paths we need to use for cesium as below

```javascript
// path.js
'use strict';

const path = require('path');
const process = require('process');

const pathBuilder = (subpath) => path.join(process.cwd(), subpath);

module.exports = {
  ... // Your current defined paths

  // The path to the CesiumJS source code
  cesiumSource: pathBuilder('node_modules/cesium/Source'),
  cesiumSourceAssets: pathBuilder('node_modules/cesium/Source/Assets'),
  cesiumSourceWidgets: pathBuilder('node_modules/cesium/Source/Widgets'),
  cesiumWorkers: pathBuilder('node_modules/cesium/Build/Cesium/Workers'),
};
```
And the most important part, to add CesiumJS into your webpack.config. Since we will use it in both dev & production, I add the configuration into the **webpack.config.common.js** file.
```javascript
// webpack.config.common.js
............... // Your current imports
const CopywebpackPlugin = require('copy-webpack-plugin');
const { DefinePlugin } = require('webpack');

// Common configs
const path = require('./path');

// Everything belongs to cesium is isolated in an object like this
const cesiumConfig = {
  resolve: {
    alias: {
      // CesiumJS module name
      cesiumSource: path.cesiumSource,
    },
  },
  amd: {
    // Enable webpack-friendly use of require in Cesium
    toUrlUndefined: true
  },
  output: {
    // Needed to compile multiline strings in Cesium
    sourcePrefix: '',
  },
  plugins: [
    // Copy Cesium Assets, Widgets, and Workers to a static directory
    new CopywebpackPlugin([ { from: path.cesiumWorkers, to: 'Workers' } ]),
    new CopywebpackPlugin([ { from: path.cesiumSourceAssets, to: 'Assets' } ]),
    new CopywebpackPlugin([ { from: path.cesiumSourceWidgets, to: 'Widgets' } ]),
    new DefinePlugin({
      // Define relative base path in cesium for loading assets
      CESIUM_BASE_URL: JSON.stringify('')
    }),
  ],
};
// Now, using the cesiumConfig in your real configuration
const config = {
  ...cesiumConfig,
  module: {
    unknownContextCritical: false,
    rules: [
      // Make sure you have below rules as default
      {
        test: /\.css$/,
        use: [ 'style-loader', 'css-loader' ]
      }, {
        test: /\.(png|gif|jpg|jpeg|svg|xml|json)$/,
        use: [ 'url-loader' ]
      }
    ],
  },

  resolve: {
    alias: {
      ...cesiumConfig.resolve.alias,
    },    
  },

  plugins: [
    ...., // your current plugins
    ...cesiumConfig.plugins,
  ],
};

module.exports = config;
```

That’s it!!! Now your project is ready to use Cesium.

### 3. Using CesiumJS with Typescript

To make sure you can use CesiumJS with correct UI everywhere, remember to add its CSS at your root component/index

```
// Import Cesium CSS
// You may need tslint:disable: to disable linting rules errors
import 'cesiumSource/Widgets/widgets.css';
```

In your component, let's callCesiumPage.tsx, import and use like below

```typescript jsx
....
// You may need to disable tslint for this require
// Notice that cesiumSource is the alias we defined in 
// webpack.config.common.js resolve.alias 
const cesium: any = require('cesiumSource/Cesium');
// Import @types/cesium to use along with CesiumJS
import { Viewer } from 'cesium';
// Import Cesium CSS if not yet added at root component
import 'cesiumSource/Widgets/widgets.css';

const Root: AnyStyledComponent =
  styled.div({
    width: '100%',
    height: '100%',
  });
const CesiumViewer: AnyStyledComponent = styled.div`
  width: 100%;
  height: 100%;
`;

export interface Props {
  ...
}

/**
 * Project page
 */
class CesiumPage extends Component<Props> {
  private cesiumContainer: RefObject<HTMLDivElement>;

  public constructor(props: Props) {
    super(props);
    this.cesiumContainer = createRef();
  }

  public componentDidMount(): void {
    if (this.cesiumContainer.current) {
      // type Viewer is from @types/cesium
      // whereas, new cesium.Viewer is from the module cesium
      const viewer: Viewer = new cesium.Viewer(this.cesiumContainer.current);
      ........
      console.log('cesium viewer = ', viewer);
    }
  }

  public render(): ReactNode {
    return (
      <Root>
        <CesiumViewer id='cesiumContainer' ref={this.cesiumContainer} />
      </Root>
    );
  }
}
export default CesiumPage;
```

### 4. Misc.

In case your project setup with Jest, you may need an additional configuration in `jest.config.js` like below

```javascript
moduleNameMapper: {
  // Your current configs...,
  'cesiumSource/Cesium$': '<rootDir>/node_modules/cesium/Source/Cesium.js',
},
transformIgnorePatterns: [
  '/node_modules/(?!(?:ol|lodash-es|cesium)/)[^/]*/.*\\.(js|jsx|ts|tsx)$',
],
```

DONE!!!

In case you have better solutions or a simpler configuration, please add a comment or share it with our community.

Thank you.

### 5. References
https://medium.com/@lhviet88/how-to-setup-cesiumjs-to-use-in-reactjs-webpack-and-typescript-a9cd8378f129


How to setup CesiumJS to use in ReactJS, Webpack, and Typescript


When people creating a new NodeJS project, let’s say using ExpressJS, the source code in default is JavaScript.  
However, what if you prefer TypeScript to keep the source code more consolidated? For example, type checking.

In that case, the source code we would like to manage in GIT will be TypeScript (.ts), instead of JavaScript (.js).

By writing source in TypeScript, we may test the localhost by one of two ways below:

*   Run localhost directly by `ts-node` without compiling from TypeScript (.ts) to JavaScript (.js)
*   Compile the code (in TypeScript) to JavaScript first and run as normal. For example: `node ./server.js`

## Options

It looks fine in your localhost, however, to deploy the project into ElasticBeanstalk (NodeJS), it may be tricky.

If you use default configuration to deploy, only code managed by GIT, it means TypeScript, will be deployed to the server. Here, to solve this problem, we have at least 4 options below:

1.  Compile the code in localhost (or CI)to get `.js` files, and add a `.ebignore` to let EB deploy the `.js` files.
2.  Manage both `.ts` and `.js` files in GIT.
3.  Compile and deploy separately. Means we will compile the code into a separate place, i.e., `dist` , copying other needed files, i.e., `package.json, public, views…`, and deploy that directory `dist`.
4.  Customize the `.ebextensions` to compile the source code on ElasticBeanstalk side before the run.

## Solution

The solution 1 looks like the simplest one, however, when using it I met error with `zip process` , which keep raising error Timestamp before 1980.

Solution 2 is not clear and not a good solution. It makes the source code hard to manage and leads to confuse in commits, because not only .ts files change, but also .js files change.

Solution 3 is complicated and request us to find some way to automate it somehow: compiling before deploying, copying necessary files, and must custom the scripts (maybe).

Solution 4 is the best for me, why?

*   GIT manages TypeScripts, the source code we write only
*   Not mess up with the localhost compilation
*   Not many configurations needed

## Steps

Assumes that you already set up an ElasticBeanstalk Application & Environment for NodeJS app. What we need is **Compiling**:

*   **After** the source code deployed (into EC2) and package **installed  
    **`**npm install**`
*   **Before** `npm start` (or anything else in NodeCommand)

What we need to do is simply add a config file into `.ebextensions` , i.e., the `source_compile.config` file below:

```
# source_compile.config
container_commands:
  compile:
    command: "./node_modules/.bin/tsc -p tsconfig.json"
    env:
      PATH: /opt/elasticbeanstalk/node-install/node-v10.15.3-linux-x64/bin/
```

The configure tells the EB to compile the nodejs source code in `/var/current/application` by command `tsc` .

The key we need to notice is the `env`, which enable **node** to be exposed in the **PATH**.

Remember to have the module `typescript` in the dependencies of your package.json file. It will provide the `tsc` in `./bin` directory of the `node_modules` after typescript installed.

```
"dependencies": {
  .......,
  "typescript": "^3.4.5",
  .......  
}
```

If there is no specific `outdir` in your tsconfig.json file, the compiled .js files will be generated in the same location of the .ts files after `command` executed.

In conclusion, to write the source code in TypeScript and deploy it to Elastic Beanstalk, simply add few lines of configurations to compile the code before Node server running will help to solve our problem.

P/s: When taking note this Typescript app Elastic Beanstalk deployment, I found a similar article which may help you. Please reference to [https://www.errietta.me/blog/typescript-circleci-aws-beanstalk/](https://www.errietta.me/blog/typescript-circleci-aws-beanstalk/)

## References
https://medium.com/@lhviet88/deploy-a-typescript-expressjs-into-elasticbeanstalk-nodejs-server-8381e00e7e52


Showing posts fromprogramming

How to setup CesiumJS to use in ReactJS, Webpack, and Typescript

Deploying a TypeScript, ExpressJS app into ElasticBeanstalk NodeJS server