OpenAI does not use any data that goes through its API to train their models.

If the concern is privacy, you can use a solution that removes PII before sending a message to the OpenAI API.

It's true that langchain allows you to work with other LLMs (Anthropic, Palm etc) but what's the strategy ? Just use several LLMs to not put all your eggs in one basket ? Just for the sake of it ?

If you really want your data to be private, you can use an open source LLM (with the limitations that ensued) and deploy it on a private server, or use the Azure OpenAI Service. LangChain is not the solution in that case.