Can we connect a source system that have no internet connectivity through Azure data factory self-hosted IR?

I have a scenario where the source system is in a private network and has no internet connectivity, But I have configured a Self-Hosted IR VM in the same network and has an active Internet connection. So my question is that, is this possible to copy data from this source (No Internet) by using Azure data factory self-hosted IR.

privacy – GitLab self-hosted instances data sharing

I’m working on a personal project with some friends and it is becoming quite large, so I’m considering to self-host a GitLab instance to help preventing the development to go to complete chaos.
However, I discovered that GitLab’s privacy policy allows GitLab to gather some data from its users and share them with third parties and I’m not OK with that, but it isn’t clear if that applies to “Self-managed” instances with Usage Ping disabled (1, 2).
Some weeks ago I wrote to the developers asking for more information, but I’m still waiting for an answer.

Normally I would just discard the option, but I found that some free (libre) software projects, e.g. The Tor Project and GNOME, use self-hosted GitLab instances. Especially the former is (should be?) extra careful about users’ privacy, so I’m not sure about what to suppose.

Can anyone provide reference (or evidence) about whether GitLab’s self-hosted instances send any data (personal or not) to GitLab or other third parties when the Usage Ping is disabled?

LowEndBoxTV: Your Own Self-Hosted DNS Using NSD

LowEndBoxTV Video on NSD If you’ve wanted to run your own DNS server but were a bit intimidated by BIND, this is the video for you. We take you through installing and configuring a 2-node DNS server setup using NSD, including setting up zone transfers and creating DNS records.

NSD is remarkably simple to use and takes a lot of the complexity out of the server part of running a DNS server.  The records are still a bit arcane but we’ll walk through the basics there.  If you need a caching nameserver, NSD isn’t for you, but if you just need a simple way to take a couple cheap VPS servers and authoritatively publish your DNS, NSD is a great solution.

This is an update to a tutorial we did quite a while ago, and goes a bit further as the video covers zone transfers.

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

aws – OpenVPN client suddenly refuses to connect to my self-hosted VPN

It’s been working fine for years, but now suddenly I’m getting a strange error. Does anyone know what problem this points to?

OpenVPN Connect Version 3.0.2 (598), error logs:

(REDACTED IPs)

5/16/2021, 2:54:17 AM OpenVPN core 3.git::d8d14e19 mac x86_64 64-bit PT_PROXY built on Apr  5 2019 07:54:59
5/16/2021, 2:54:17 AM Frame=512/2048/512 mssfix-ctrl=1250
5/16/2021, 2:54:17 AM EVENT: RESOLVE ⏎5/16/2021, 2:54:17 AM UNUSED OPTIONS
4 (nobind) 
18 (sndbuf) (0) 
19 (rcvbuf) (0) 
22 (verb) (3) 
31 (CLI_PREF_ALLOW_WEB_IMPORT) (True) 
32 (CLI_PREF_BASIC_CLIENT) (False) 
33 (CLI_PREF_ENABLE_CONNECT) (True) 
34 (CLI_PREF_ENABLE_XD_PROXY) (True) 
35 (WSHOST) (35.158.212.188:443) 
36 (WEB_CA_BUNDLE) (-----BEGIN CERTIFICATE----- MIIDBj(REDACTED)BgkqhkiG...) 
37 (IS_OPENVPN_WEB_CA) (1) 
38 (ORGANIZATION) (OpenVPN Inc) 
⏎5/16/2021, 2:54:17 AM Contacting (REDACTED):1194 via UDP
⏎5/16/2021, 2:54:17 AM EVENT: WAIT ⏎5/16/2021, 2:54:17 AM Connecting to (REDACTED):1194 ((REDACTED)) via UDPv4
⏎5/16/2021, 2:54:17 AM Tunnel Options:V4,dev-type tun,link-mtu 1558,tun-mtu 1500,proto UDPv4,comp-lzo,keydir 1,cipher AES-256-CBC,auth SHA1,keysize 256,tls-auth,key-method 2,tls-client
⏎5/16/2021, 2:54:17 AM Creds: Username/Password
⏎5/16/2021, 2:54:17 AM Peer Info:
IV_VER=3.git::d8d14e19
IV_PLAT=mac
IV_NCP=2
IV_TCPNL=1
IV_PROTO=2
IV_LZO_STUB=1
IV_COMP_STUB=1
IV_COMP_STUBv2=1
IV_HWADDR=f4:5c:89:cd:69:cd

⏎5/16/2021, 2:54:17 AM EVENT: CONNECTING ⏎5/16/2021, 2:54:17 AM SSL Handshake: TLSv1.2/TLS-ECDHE-RSA-WITH-AES-256-GCM-SHA384
⏎5/16/2021, 2:54:17 AM EVENT: GET_CONFIG ⏎5/16/2021, 2:54:17 AM Session is ACTIVE
⏎5/16/2021, 2:54:17 AM Sending PUSH_REQUEST to server...
⏎5/16/2021, 2:54:17 AM OPTIONS:
0 (explicit-exit-notify) 
1 (topology) (subnet) 
2 (route-delay) (5) (30) 
3 (dhcp-pre-release) 
4 (dhcp-renew) 
5 (dhcp-release) 
6 (route-metric) (101) 
7 (ping) (12) 
8 (ping-restart) (50) 
9 (compress) (stub-v2) 
10 (redirect-gateway) (def1) 
11 (redirect-gateway) (bypass-dhcp) 
12 (redirect-gateway) (autolocal) 
13 (route-gateway) (172.27.232.1) 
14 (dhcp-option) (DNS) (172.31.0.2) 
15 (register-dns) 
16 (block-ipv6) 
17 (ifconfig) (172.27.232.5) (255.255.248.0) 
18 (peer-id) (1) 
19 (auth-token) ...
20 (cipher) (AES-256-GCM) 

⏎5/16/2021, 2:54:17 AM Session token: (redacted)
⏎5/16/2021, 2:54:17 AM Server has pushed compressor COMP_STUBv2, but client has disabled compression, switching to asymmetric
⏎5/16/2021, 2:54:17 AM PROTOCOL OPTIONS:
  cipher: AES-256-GCM
  digest: SHA1
  compress: COMP_STUBv2
  peer ID: 1
⏎5/16/2021, 2:54:17 AM TunPersist: long-term session scope
⏎5/16/2021, 2:54:17 AM TunPersist: new tun context
⏎5/16/2021, 2:54:17 AM CAPTURED OPTIONS:
Session Name: 35.158.212.188
Layer: OSI_LAYER_3
MTU: 1500
Remote Address: (REDACTED)
Tunnel Addresses:
  172.27.232.5/21 -> 172.27.232.1
Reroute Gateway: IPv4=1 IPv6=0 flags=( ENABLE REROUTE_GW AUTO_LOCAL DEF1 BYPASS_DHCP IPv4 )
Block IPv6: yes
Route Metric Default: 101
Add Routes:
Exclude Routes:
DNS Servers:
  172.31.0.2
Search Domains:

⏎5/16/2021, 2:54:17 AM SetupClient: transmitting tun setup list to /var/run/ovpnagent.sock
{
    "config" : 
    {
        "iface_name" : "",
        "layer" : "OSI_LAYER_3",
        "tun_prefix" : false
    },
    "tun" : 
    {
        "adapter_domain_suffix" : "",
        "block_ipv6" : true,
        "dns_servers" : 
        (
            {
                "address" : "172.31.0.2",
                "ipv6" : false
            }
        ),
        "layer" : 3,
        "mtu" : 1500,
        "remote_address" : 
        {
            "address" : "(REDACTED)",
            "ipv6" : false
        },
        "reroute_gw" : 
        {
            "flags" : 315,
            "ipv4" : true,
            "ipv6" : false
        },
        "route_metric_default" : 101,
        "session_name" : "(REDACTED)",
        "tunnel_address_index_ipv4" : 0,
        "tunnel_address_index_ipv6" : -1,
        "tunnel_addresses" : 
        (
            {
                "address" : "172.27.232.5",
                "gateway" : "172.27.232.1",
                "ipv6" : false,
                "metric" : -1,
                "net30" : false,
                "prefix_length" : 21
            }
        )
    }
}
POST unix://(/var/run/ovpnagent.sock)/tun-setup : E_CONNECT HTTPCore Asio handle_unix_connect: No such file or directory
⏎5/16/2021, 2:54:17 AM TUN Error: ovpnagent: communication error
⏎5/16/2021, 2:54:17 AM Client exception in transport_recv: tun_exception: not connected
⏎5/16/2021, 2:54:17 AM EVENT: ASSIGN_IP ⏎5/16/2021, 2:54:17 AM EVENT: TUN_SETUP_FAILED ovpnagent: communication error⏎5/16/2021, 2:54:17 AM EVENT: DISCONNECTED ⏎5/16/2021, 2:54:19 AM Raw stats on disconnect:
  BYTES_IN : 3120
  BYTES_OUT : 2182
  PACKETS_IN : 8
  PACKETS_OUT : 7
  TUN_SETUP_FAILED : 1
⏎5/16/2021, 2:54:19 AM Performance stats on disconnect:
  CPU usage (microseconds): 10806780
  Network bytes per CPU second: 490
  Tunnel bytes per CPU second: 0
⏎

LowEndBoxTV: Create Your Own Self-Hosted Disposable Webmail with Inbucket

Services like Mailinator allow anyone to setup disposable webmail accounts for easy filtering of disposable emails. If you are hesitant to give your email to web sites or organizations because you know they’re going to sign you up for mailing lists, this approach allows you an unlimited number of instantly-generated emails to preserve your sanity. Unfortunately, marketers are wise to Mailinator and many sites now disallow signups with Mailinator.com emails. However, you can self-host your own Mailinator-like service with an excellent FOSS product called inbucket. In this tutorial, we walk you through setup from start to finish on a freshly-installed Debian VPS.


As we launch our channel, we would appreciate your likes, shares, and above all subscriptions.  We’re also interested to know what kind of content you’d like to see – please comment below!

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

Is there any self-hosted “notes only” CalDav server or exchange server?

I’m trying to build a Linux server that replace iCloud notes sync on iPhone and Mac, after google I know that the notes sync depends on CalDav.

So is there any open source server side app that provides iCloud-like notes sync on Linux? With or without mail, calendar… doesn’t matter.

I’m open for any suggestions, thank you all.

Take Control of Your Passwords with Self-Hosted Bitwarden

There are many password managers on the market today. Most are vendor-hosted, such as LastPass, which puts you in the position of trusting the vendor’s security (on both the client and storage side). Some offer self-hosted options, but generally require Dropbox or other cloud storage.  There are also standalone apps where you engineer any synchronization yourself, but not all provide the creature comforts of cross-device synchronization and browser fillers.

In this tutorial, we’ll look at Bitwarden, a password manager that offer self-hosted options, and walk through setting it up.

First, take note of the somewhat steep hosting requirements: 4G of RAM.  You may be able to cheat a little if you’re the only user, but you’re not going to host this on a 128MB system.

Also, note that Bitwarden is docker-based, which has implications for those wanting to host on OpenVZ.  But really, it’s 2020, folks.

Before you begin, head over to https://bitwarden.com/host/ and enter your email address.  This will provision you an installation ID and installation key, which you’ll need to install BitWarden

With those out of the way, let’s get down to typing commands. Install some prereqs:

apt install -y apt-transport-https ca-certificates wget gnupg2 software-properties-common curl

Now we’ll add the docker repo:

wget -O - https://download.docker.com/linux/debian/gpg | apt-key add
echo "deb (arch=amd64) https://download.docker.com/linux/debian buster stable" > /etc/apt/sources.list.d/docker.list
apt update
apt install -y docker-ce docker-ce-cli containerd.io

Bitwarden wants a later/greater docker-compose than is in the repo as of this time of writing, so I’ll manually download it:

curl -L "https://github.com/docker/compose/releases/download/1.26.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod 755 /usr/local/bin/docker-compose

Now you can start and enable docker:

cd systemctl start docker
systemctl enable docker

Let’s make sure docker works:

root@bitwarden:~# docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
0e03bdcc26d7: Pull complete 
Digest: sha256:d58e752213a51785838f9eed2b7a498ffa1cb3aa7f946dda11af39286c3db9a9
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.

To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash

Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/

For more examples and ideas, visit:
https://docs.docker.com/get-started/

One final step is to install the mail system of your choice.  If you don’t have a choice, postfix is easy to use.

apt install postfix

Select “Internet Site” and enter the FQDN of your server (in this case, bitwarden.lowend.party).

I host in /bitwarden, but you can put your install anywhere you like.

mkdir /bitwarden
cd /bitwarden
curl -Lso bitwarden.sh https://go.btwrdn.co/bw-sh && chmod +x bitwarden.sh
./bitwarden.sh install

There’s a lot out output and status messages.  You will need to answer some questions:

(!) Enter the domain name for your Bitwarden instance (ex. bitwarden.example.com): bitwarden.lowend.party
(!) Do you want to use Let's Encrypt to generate a free SSL certificate? (y/n): y
(!) Enter your email address (Let's Encrypt will send you certificate expiration reminders): raindog308@raindog308.com
(!) Enter your installation id (get at https://bitwarden.com/host): (you got this above)
(!) Enter your installation key: (you got this above)

We need to configure mail parameters before we start Bitwarden.  To do this, edit /bitwarden/bwdata/env/global.override.env and change as follows:

globalSettings__mail__replyToEmail=no-reply@bitwarden.lowend.party
globalSettings__mail__smtp__host=bitwarden.lowend.party
globalSettings__mail__smtp__port=25
globalSettings__mail__smtp__ssl=false
globalSettings__mail__smtp__username=
globalSettings__mail__smtp__password=

This assumes you’re running a stock postfix configuration.  You can also configure to use a different mail host, use SSL, login with a username/password, etc.

Once this step is complete, you’re ready to run Bitwarden.  Let’s do it through systemd, so that if our VM restarts for any way, Bitwarden will start on boot.

Place the following in /etc/systemd/system/bitwarden.service:

(Unit)
Description=Bitwarden
Requires=docker.service
After=docker.service

(Service)
ExecStart=/bitwarden/bitwarden.sh start
ExecStop=/bitwarden/bitwarden.sh stop

(Install)
WantedBy=default.target

Now enable and start Bitwarden:

systemctl daemon-reload
systemctl enable bitwarden
systemctl start bitwarden

Bitwarden can take a minute or two to startup because it has to pull docker images, etc.  BTW, yes, that’s Microsoft SQL Server (Express Edition) running under the covers.

Before we login, we want to configure postfix to accept mail from Bitwarden.  Because Bitwarden is running via docker, when we tell it to send email to localhost, it’s not the same localhost Postfix is thinking of.  Let’s see where it’s running.

# docker network ls
NETWORK ID          NAME                DRIVER              SCOPE
d9cc1f3a6b10        bridge              bridge              local
f98a474288df        docker_default      bridge              local
87cb796a4769        docker_public       bridge              local
13f31967b4d1        host                host                local
c15a9550aee7        none                null                local

If you were to look in /bitwarden/bwdata/docker/docker-compose.yml, you’d see that the ‘api’ container uses the ‘default’ and ‘public’ networks.  After issuing

docker network inspect 87cb796a4769

I see that the docker API container is running on 172.19.0.3.

Edit /etc/postfix/main.cf to add the part in bold:

mynetworks = 127.0.0.0/8 (::ffff:127.0.0.0)/104 (::1)/128 172.19.0.0/16

Then type

postfix reload

Once Bitwarden is up and running, browse to your hostname.  In this case, https://bitwarden.lowend.party.

Click Create Account.

Fill out the form. Note that you’re not creating an account on bitwarden.com but rather on your server.

Now login to your account.

You can click Verify Email and an email verification will be sent to you.

Let’s add an item.  Click Add Item.

Click Save and it’s now ready for synchronization everywhere.

Configuring Your Client Devices

I’m going to walk through setting up the Chrome extension.

Once setup, click on the Bitwarden icon and the setup screen will drop down.

If you attempt to login there, you’ll be logging into Bitwarden.com.  Instead, click the gears icon.

Enter your server URL.  Then save and login with your credentials.

Your synchronized vault will appear.

The setup on phones, etc. is virtually identical as it’s the same panels and forms.  Remember click the gear icon to enter your custom server URL.

Let’s use this login.  I browse to the world’s greatest web site and click Sign In.  When the Sign In box pops up, I hit control-shift-L (command-shift-L on Mac) and my credentials are autofilled.

Right now, anyone who comes across your Bitwarden install can click Create Account and set themselves up.  They won’t have access to your logins (unless they know your login and master password) but you don’t want people setting up accounts.

To prevent this, edit /bitwarden/bwdata/env/global.override.env and set disableUserRegistration to true:

globalSettings__disableUserRegistration=true

This toggles account to creation to false.  Note that the button will still appear in the GUI but people won’t be able to create accounts.

To backup Bitwarden, you should

  • Shutdown Bitwarden to ensure the database is consistent
  • Backup /bitwarden/bwdata
  • Restart Bitwarden

Should you need to restore a backup, follow this Bitwarden doc.

 

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

Server running on Windows 10 as a service, OWIN self-hosted with WebAPI endpoints making an SSL connection without configuring a private key

I have created a prototype application the runs on Windows 10 that communicates with a server (described in the title) running as a service on a different system and successfully got SSL working but I’m missing something because I never seemed to generate any keys. I only just self-taught myself how to use PowerShell to create the self-signed certificate (so bare with me) using New-SelfSignedCertificate but I have very little insight into why this worked at all. From my very limited understanding a private key on the server is absolutely required for SSL to work, and I never associated one with my certificate… but it’s working and I think I’m fooling myself.

All I did was create the certificate with New-SelfSignedCertificate -Subject "CN=My Server Name"
Then add the binding with netsh http add sslcert certhash=<the thumbprint> appid={the app id}
And add the reservation for the service with netsh http add urlacl url="https://+:<my port number>/" user='NT AUTHORITYLocalService'

This all works, but I’m not sure I understand how because I never configured a private key. I know it must need to be expanded upon somehow to make it more secure because every single tutorial out there talks about making the certificate trustworthy and assigning a private key to the cert… but I’ve done none of that and it still appears to work. Granted, the client is not a web browser and will never care about the authenticity of the server because of the context in which this particular application will run (It’s to connect scientific components together in a lab, and there’s almost no exposure outside the network).

What am I missing here? Did Windows 10 provide a private key anyway? How can it really be encrypted if I didn’t need to specify these things? I also never moved the certificate to a trusted store of any kind because it frankly didn’t seem necessary since the client is just a proprietary front-end application.