bash – Ansible to automate dev environment setup

This is a follow-up to Bash script to automate dev environment setup.

In that question I’d thrown together a (sloppy) shell script to automatically setup my development environment. One of the answers suggested using Ansible and after a bit of reading realized it would help me with some configuration of remote servers as well so I decided to give it a go.

Below is playbook that sets up my dev environment the same way as the original bash script but hopefully a bit cleaner. I’m planning on using Ansible to setup a new CI/CD pipeline in a reproducible way as a replacement for the github -> dockerhub -> manual deployment so this is really testing the waters with Ansible before moving on to that.

Right now it all I need to do is clone the repo the two files are in and then run bootstrap.sh and everything gets set up from there.

Any/all pointers appreciated!

bootstrap.sh:

sudo apt update && sudo apt -y upgrade
sudo apt install -y ansible
mv AnsibleDevEnv/setup.yml ~/
ansible-playbook setup.yml
. .bash_profile

And then the Ansible playbook:

---
- name: Dev Setup
  hosts: localhost
  vars:
  
    folders:
    - go
    - python
    - js
    - pemKeys
    
    downloads:
      url:
      - https://deb.nodesource.com/setup_14.x
      - https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh
      - https://storage.googleapis.com/golang/getgo/installer_linux

      sudo_files:
      - setup_14.x
       - Anaconda3-2020.02-Linux-x86_64.sh
        
      user_files:
      - installer_linux
      
    keys:
      - https://packages.microsoft.com/keys/microsoft.asc
      - https://download.docker.com/linux/debian/gpg
      
    repos:
      - deb (trusted=yes arch=amd64) https://download.docker.com/linux/debian {{ docker_version.stdout }} stable
      - deb (arch=amd64) https://packages.microsoft.com/repos/vscode stable main
      
    packages:
      - apt-transport-https 
      - ca-certificates 
      - gnupg2 
      - software-properties-common
      - libgl1-mesa-glx 
      - libegl1-mesa 
      - libxrandr2 
      - libxrandr2 
      - libxss1 
      - libxcursor1 
      - libxcomposite1 
      - libasound2 
      - libxi6 
      - libxtst6
      - libpq-dev 
      - python3-dev
      - python3-pip
      - protobuf-compiler
      - apt-transport-https
      - code
      - nodejs
      - postgresql-11
      - docker-ce
      
    node_lib:
      - react
      - react-scripts
      - react-dom
      
    go_get:
      - go get github.com/lib/pq
      - export
      - GO111MODULE=on go get github.com/golang/protobuf/protoc-gen-go
      - GO111MODULE=on go get -u google.golang.org/grpc
      
    pip:
      - psycopg2
      
    git_config:
      name: 
        - user.name
        - user.email
        - color.ui
      value:
        - cmelgreen
        - cmelgreen@gmail.com
        - true

  tasks:
    - name: make folders
      file:
        path: './{{ item }}'
        mode: 0755
        state: directory
      with_items: '{{ folders }}'
      
    - name: install rpm
      apt:
        name: rpm
        state: latest
        update_cache: yes
      become: yes
        
    - name: add keys
      apt_key:
        state: present
        url: '{{ item }}'
      with_items: '{{ keys }}'
      become: yes
      
    - name: save docker version to variable
      shell: lsb_release -cs
      register: docker_version
      
    - name: add repositories
      apt_repository: 
        repo: '{{ item }}'
        state: present
      with_items: '{{ repos }}'
      become: yes
        
    - name: download files
      get_url: 
        url: '{{ item }}'
        dest: .
        mode: +x
      with_items: '{{ downloads.url }}'
        
    - name: run as root downloads
      command: './{{ item }}'
      with_items: '{{ downloads.sudo_files }}'
      become: yes
      
    - name: run as user downloads
      command: './{{ item }}'
      with_items: '{{ downloads.user_files }}'

    - name: add source ./.bashrc to .bash_profile
      lineinfile:
        path: ./.bash_profile
        line: 'source ./.bashrc'
        
    - name: install packages
      apt: 
        name: '{{ packages }}'
        state: latest
        update_cache: yes
      become: yes
       
    - name: set docker permissions
      file:
        path: /var/run/docker.sock
        mode: 0666
      become: yes

    - name: install react
      npm:
        name: '{{ item }}'
        global: yes
        state: latest
      with_items: '{{ node_lib }}'
      become: yes

    - name: go get some libraries
      shell: '. ~/.bash_profile && {{ item }}'
      args:
        executable: /bin/bash
      with_items: '{{ go_get }}'


    - name: pip some stuff conda has a hard time with
      pip:
        name: '{{ pip }}'

cron setup on FreeBSD instead of Cpanel

I am trying to install a cron with the instructions received which does not match with my current hoster.
I received as instruction:
” Enter the command “curl -s https: // /cron.php?token= ” in the blank field next to the Command field.
(https: // /cron.php?token= is the URL we received in the first part)”
Token is with a number sequence

While my hoster is on FreeBSD with a personalized panel:
/usr/local/bin/php -f /home/username/bin/example.php> /dev/null 2>&1

I don’t know how to make the configuration changes to get my cron to work properly.

Thank you for your help
Best regards

CTF setup for debugging heap exploits

I’m hope this is the right StackExchange to ask the following question

I’m currently studying binary heap exploitation (mainly the glibc (ptmalloc2) implementation), for CTF competitions. The problem I’m facing is debugging challenges designed for a certain glibc version. I know there are some differences between (for example) libc 2.27 and libc 2.31 that can make the exploitation of certain vulnerabilities harder (or impossible) to exploit on newer versions.

This is a template script I usually like to use (with pwntools):

#!/usr/bin/env python3

from pwn import *

BINARY = ""
LIBC = ""
HOST = ""
PORT = 9999

exe = ELF(BINARY)
if LIBC != "":
    libc = ELF(LIBC, checksec=False)

rop = ROP(exe)

context.binary = exe
context.log_level = "debug"


if "remote" in sys.argv:
    io = remote(HOST, PORT)
else:
    io = gdb.debug((BINARY), gdbscript="""
         b main
    """
    )


io.interactive()

This makes debugging locally extremely easy and fast IMO.

I’d like to have a method to easily be able to debug programs using the same (or kind of the same) script as above and using the right glibc (and ld as both are required to be the same version) version without messing with my system’s library.

This might be a rather dumb question but I’ve failed to find any good answer googling around!

Please share your setups for these kind of debugging, I’ll appreciate any help!

Setup your HYIP in a few simple clicks – HYIPs

Don’t depend only on your regular income, make a new source of income. Zeligz HYIP is giving an opportunity to start your own HYIP business at the lowest cost. Buy HYIP Script and setup your HYIP Investment Program in a few simple clicks. It has many advanced features that will help you in managing your business and also, provide powerful security to your online investment program. Get detailed information about it at Zeligz Web Store.
 

smtp – Not found error in Mail Server setup (hMailServer)

I have set up the following configurations in Domain and server settings but I receive this error when trying http://mail.example.com :

Not Found
HTTP Error 404. The requested resource is not found.

Setting:

  1. added A record with mail name pointing to server ip.
  2. added MX record with empty` name pointing to server ip (also tested pointing to mail.example.com).
  3. Installed hMailServer on server.
  4. connected to hMailServer using localhost and administrator account.
  5. added example.com in domains.
  6. added mail.example.com in Setting > SMTP > Delivery of E-mail > local host name.
  7. added port 25 to both firewall incoming and outging rules.

Foot note: the domain example.com is properly defined on IIS and is working when I try http://example.com

multistore – Magento 2.3.3 Newbie – How To Setup Multiple Stores in Multi-Domain?

I’m sorry if this has been asked/answered in a different way of wording it. My apologies.

I have, what seems to be an EXTREMELY simple problem that – no matter how I search it – seems to come up with different solutions. Again, please excuse my ignorance of the solution.

BACKGROUND:
The only reason I switched to Magento is because of Multi-Domain support. I currently have a site up that houses over 400,000 SKUs and I want to be able to move those over to one backend and serve different descriptions/prices on each domain. I am hosted currently on Cloudways AWS server and have the DNS pointing to the correct IP addresses as verified by ‘whatsmydns.com’.

PROBLEM:

When navigating to ‘PCS.COM’ I get ‘PCS.COM’ to appear as ‘PCS.COM/DEFAULT’
When navigating to ‘CRESCOM.COM’ I get ‘PCS.COM’ to appear as ‘PCS.COM/DEFAULT’
When navigating to ‘RRESCOM.COM’ I get ‘PCS.COM’ to appear as ‘PCS.COM/DEFAULT’

Id like each domain to point to the correct domain…

I was given instruction from Cloudways that says the following:

1- change the webroot to /pub
2- Fetch the mage run code and mage run type of your stores.
3- Verify the domains in the store-> configurations -> web. Every store should have its own domain, not using the default value.
4- Add the following rules inside the /pub/index.php before $params = $_SERVER;

The code that was provided says:

switch ($_SERVER('HTTP_HOST')) {

case 'abc.com':
                $mageRunCode = 'abc_base';
                $mageRunType = 'website';
                break;

        case 'xyz.com':
                $mageRunCode = 'xyz_base';
                $mageRunType = 'website';
                break;


        default:
                $mageRunCode = 'base';
                $mageRunType = 'website';
                break;
                        }

TRIED:

  • Updating the pub/index.php file to the recommended settings.
  • Setting up all Stores > Config > Web settings to the correct values
  • Deleting Server & Reinstalling Magento 2.3.3
  • Researching to the fullest extent I can. No matter where I look there are different ways to redirect traffic to the correct domain.

FRUSTRATED:
I don’t know why/what/how to manage these redirections work. It seems like it should be a simple deal to do but I am at a loss – any help would be appreciated.

ADDITIONAL QUESTIONS:

  • In regards to $mageRunCode – I don’t know what code to put in. The Website, Store, or StoreView?
  • In regards to $mageRunType – I don’t know what code to put in. Store or Store View…..Or Web Site?

FURTHERMORE:
I’ve seen ‘solutions’ that mention editing the .htaccess file but from my research this would not be optimal because an update to M2 MAY edit that file to make it useless. True?

THANK YOU:
I just want to ‘Thank You’ for you time, assistance, and patience with me with this issue. For you developers out there I have nothing but respect. //// I just don’t know (YET) how to do this operation.
Thank You..Thank You..Thank You.

bitcoind – How can I setup Bitcoin to be anonymous with Tor?

This is not a thorough schooling on Tor and only shows how to configure it to work together with Bitcoin Core.

Bitcoin Core includes Tor integration

When Tor is correctly setup on your system, Bitcoin Core automatically identifies Tor and creates an anonymous service. Little configuration is required to be ‘off the grid’ and, just a tiny bit more to be completely anonymous if that is important to you, with none of your Bitcoin traffic reaching out onto the public internet.

Using these steps you can be anonymous in only five minutes.

With the full privacy setup, transactions will of course still be broadcast but will only be broadcast actually onto the public internet by other Bitcoin nodes. With the standard ‘off-the-grid’ Tor setup, your Bitcoin traffic will be routed through the anonymous Tor network before reaching the public internet and other Bitcoin nodes on and off the Tor network to be effectively untraceable.

Setting Up Bitcoin Core and Tor

These instructions work on Fedora 23>29 and assume a default setup of Bitcoin Core v0.15.1 and Tor v0.2.7.1 or newer (and have been tested to work with Bitcoin Core v0.16.0 on Fedora 27 with Tor v0.3.1.9). Fedora is a modern operating system that will run on most standard modern hardware. The configuration is the same on Windows, but the instructions are different. There are some instructions for setting up Tor on Windows here.

Further instructions for other *nix based systems are available here. NOTE: You do not need to configure your Tor client as a relay or exit node for Tor to operate, so you can skip the step for ‘Put the configuration file /etc/tor/torrc place:’ in that guide. You will still need to use all of the following steps in this guide.

  1. Setup Tor

    1. Install the tor package:

      sudo dnf install tor
      
    2. Start the tor daemon and make sure it starts at boot:

      sudo systemctl enable tor
      sudo systemctl start tor
      
  2. Figure out where your torrc file is (/etc/tor/torrc is one possibility).

  3. Open the torrc file to edit:

    xhost +local: ## skip if earlier than v29 only needed for Wayland
    sudo gedit /etc/tor/torrc
    

    or

    sudo nano /etc/tor/torrc
    
  4. Add these lines to your torrc (or ensure that they are uncommented):

    ControlPort 9051
    CookieAuthentication 1
    CookieAuthFileGroupReadable 1
    
  5. You need to figure out what group tor is using. On Fedora 23 it is toranon. Run the following command:

    ps -eo user,group,comm |egrep 'tor' |awk '{print "tor group: " $2}'
    
  6. You need to figure out what user bitcoind or bitcoin-qt is running as. Run the following command while Bitcoin is running:

    ps -eo user,group,comm |egrep 'bitcoind|bitcoin-qt' |awk '{print "Bitcoin user: " $1}'
    
  7. Run the following command as root, which adds your Bitcoin user to the tor group. Replace TOR_GROUP and BITCOIN_USER with the actual information found above:

    sudo usermod -a -G TOR_GROUP BITCOIN_USER
    

If you don’t modify any other settings, Bitcoin Core will usually connect over the regular Internet, but will also allow connections to and from the hidden Tor service.

  1. So that Bitcoin Core wil only connect via Tor (for standard ‘off-the-grid’ setup), add these lines to bitcoin.conf. In Bitcoin Core, go to Settings -> Options -> Open Configuration File. Bitcoin Core uses Tor stream isolation by default:

    proxy=127.0.0.1:9050 #If you use Windows, this could possibly be 127.0.0.1:9150 in some cases.
    listen=1
    bind=127.0.0.1
    
  2. (optional) If you like, you can add some onion service peer nodes to connect to. This will help especially if you do all of the following optional configurations. Add the following lines to your bitcoin.conf file. Bitcoin Core will only connect to a maximum of eight of these at any one time randomly, depending which ones are online:

    #Add seed nodes
    seednode=wxvp2d4rspn7tqyu.onion
    seednode=bk5ejfe56xakvtkk.onion
    seednode=bpdlwholl7rnkrkw.onion
    seednode=hhiv5pnxenvbf4am.onion
    seednode=4iuf2zac6aq3ndrb.onion
    seednode=nkf5e6b7pl4jfd4a.onion
    seednode=xqzfakpeuvrobvpj.onion
    seednode=tsyvzsqwa2kkf6b2.onion
    
    #And/or add some nodes
    addnode=gyn2vguc35viks2b.onion
    addnode=kvd44sw7skb5folw.onion
    addnode=nkf5e6b7pl4jfd4a.onion
    addnode=yu7sezmixhmyljn4.onion
    addnode=3ffk7iumtx3cegbi.onion
    addnode=3nmbbakinewlgdln.onion
    addnode=4j77gihpokxu2kj4.onion
    addnode=546esc6botbjfbxb.onion
    addnode=5at7sq5nm76xijkd.onion
    addnode=77mx2jsxaoyesz2p.onion
    addnode=7g7j54btiaxhtsiy.onion
    addnode=a6obdgzn67l7exu3.onion
    addnode=ab64h7olpl7qpxci.onion
    addnode=am2a4rahltfuxz6l.onion
    addnode=azuxls4ihrr2mep7.onion
    addnode=bitcoin7bi4op7wb.onion
    addnode=bitcoinostk4e4re.onion
    addnode=bk7yp6epnmcllq72.onion
    addnode=bmutjfrj5btseddb.onion
    addnode=ceeji4qpfs3ms3zc.onion
    addnode=clexmzqio7yhdao4.onion
    addnode=gb5ypqt63du3wfhn.onion
    addnode=h2vlpudzphzqxutd.onion
    addnode=n42h7r6oumcfsbrs.onion:4176
    addnode=ncwk3lutemffcpc4.onion
    addnode=okdzjarwekbshnof.onion
    addnode=pjghcivzkoersesd.onion
    addnode=rw7ocjltix26mefn.onion
    addnode=uws7itep7o3yinxo.onion
    addnode=vk3qjdehyy4dwcxw.onion
    addnode=vqpye2k5rcqvj5mq.onion
    addnode=wpi7rpvhnndl52ee.onion
    

If you additionally want Bitcoin Core to only connect out to Tor hidden services and not even to connect to IPv4/IPv6 nodes on the public internet via the Tor network proxy:

  1. (optional) Also add this to bitcoin.conf for full anonymity (not particularly recommended)*:

    onlynet=onion
    

*Note: Bitcoin Core will still query for peer addresses via DNS lookup if low on addresses. This also can be disabled using the next option. However, it is possible your node may not be able to find any other nodes to connect to.

*Note: Bitcoin Core v0.15.1 currently seems to make some outbound IPv4 connections at node startup even when onlynet=onion, none have been observed after initial startup. These connections should be made via your onion proxy, however, using the next option has been observed to prevent them.

  1. (optional) (advanced) If you also want to disable DNS lookup to query for peer addresses then also add the following to bitcoin.conf (not particularly recommended) note: if you use this option your node may be unable to find peers until you add some good peers with the addnode= parameter.:

    dnsseed=0
    dns=0
    
  2. Restart tor:

    sudo systemctl stop tor
    sudo systemctl start tor
    
  3. Log out of your user, log back in (this is so that your new user group permissions are effective, I do not know what user you are running Bitcoin Core on).

  4. Restart Bitcoin Core. Since Tor version 0.2.7.1 and newer the Bitcoin Core GUI version called bitcoin-qt automatically registers your Tor hidden service and makes it reachable on the onion network. For the command line version of Bitcoin Core, bitcoind, add the following parameter to your command line:

    >bitcoind -listenonion
    

No port forwarding is necessary for everything to work with Tor including incoming connections via the Tor hidden service, you do not need to forward any ports for Bitcoin Core or Tor for this.

If you want your Bitcoin node still publicly reachable via the public internet for incoming connections you will still need to forward port 8333 for Bitcoin Core.

Checking everything is working

There are only two things to check that all is working. Checking peer info in the debug window of bitcoin-qt, you should see that connections to IPv4/IPv6 peers now have some extra connected ‘via’ info along with the peer address when you click on a peer. Onion addresses only route via Tor.

Checking the same thing via console or CLI for getnetworkinfo, you should see for each network type the proxy info and, checking with getpeerinfo you should see that the addrlocal info is a remote address for each peer. Onion peers do not have addrlocal and just have their onion service name for addr.

The second thing to check is that your onion service for inbound Tor connections is up and all configuration is in place. Have a look in your debug.log file, you should see a few entries after the most recent node restart that match the following:

2018-02-10 06:31:48 InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
2018-02-10 06:31:48 InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
...
2018-02-10 06:32:13 Bound to 127.0.0.1:8333
...
2018-02-10 06:32:13 torcontrol thread start
2018-02-10 06:32:13 tor: Got service ID {onion}, advertising service {onion}.onion:8333
2018-02-10 06:32:13 AddLocal({onion}.onion:8333,4)

The advertising service information is your onion service address.

In the debug.log, connections to onion peers will only look like the following but still show up in the peers tab of the debug window on bitcoin-qt:

2018-02-10 06:34:07 receive version message: /Satoshi:0.15.1/: version 70015, blocks=508469, us=(::):0, peer=7

It is not necessary to configure port forwarding on your modem/router for Tor to operate. If you are behind a restrictive firewall it may be necessary to configure outbound connections to allow Tor to connect out to other Tor nodes. Tor can be configured to only connect out using port 80/443 if that helps. See Appendix 1 – Monitoring Tor for nyx and access to full Tor configuration options.

It is difficult to be completely anonymous since the sender and the receiver know, however, you can obfuscate your transaction origin so that your data cannot be traced by IP address without breaching the Tor network. Do some research, onlynet=onion is more secure.

Additionally, there has been research(1)(2) done on graphing the blockchain in an attempt to trace all BTC to their origin, potentially identifying source<-wallet<-purchase and depending on the combination of UTXOs potentially identifying wallet balances or wallet balance subsets. Data linkage is a privacy issue we may all be concerned about, this article discusses the use of bitcoin mixers, and this series of tweets.

Done! Enjoy being anonymous!

Appendix 1 – Monitoring Tor

You can monitor (and further tweak/break) Tor using nyx.

There are several installation methods available. On Fedora 27:

sudo dnf install nyx

To start nyx simply type nyx in the console and it will connect to Tor if it is running.

Footnotes

Once correctly configured, most synchronisation issues are to do with your hardware. See this answer for more information.

There are more configuration options available, and additional ways you can support the Tor network. Please see the several pages available here for information.

Thanks to en.bitcoin.it for your excellent guide that got me started on this.

Additional information is available from the bitcoin project here.

For an even higher level of anonymity, it is possible to configure Tor as a DNS resolver and, configure your system network configuration to use Tor to resolve DNS queries.

*by default, Tor will participate in the Tor network.