concurrency – Managing concurrent TCP connections with Go, Docker and Kubernetes

I need to consume several APIs concurrently. In order to do that I decided to containerize each API client code and manage them using Kubernetes. Some of those APIs need to be “walked”. They basically have this one endpoint from which you get more endpoints and so on. These “trees” are not that deep, but are constantly changing so hard coding them is not an option. Main challenge is limiting the number of open TCP connections. If I just spawn a goroutine as soon as I get the endpoints program will die because of too many open file descriptors. So the obvious solution is to implement a worker pool. The question is how big should it be?

As far as my understanding of these technologies goes, whole Kubernetes cluster has it’s own limit of open TCP connections, so the sum of connections on each container should not exceed it. And I should also be able to set maximum number of TCP connections for Docker containers. I think that somehow getting the system limit and then making a worker pool of that size makes sense.

dungeons and dragons – Are there rules and/or lore in any edition of D&D for managing time-travel paradoxes?

This product (which is long out-of-print but purchasable in PDF form) includes rules for time manipulation, paradoxes, etc. It includes spells explicitly designed to cause paradoxes, such as the 3rd-level spell minor paradox, the more powerful 6th-level spell paradox, and the immensely powerful 9th-level spell major paradox. Chapter 6 is all about the consequences of using such magic and “guidelines” for DMs “to solve any problems that might crop up.” (See p. 64). Chapter 3 also lays out ideas for time-traveling adventures and some of the player-facing difficulties that result from altering history.

As for lore, the only setting-specific lore on time travel of which I am aware comes from Forgotten Realms, and is detailed in this Q&A. Chronomancer, meanwhile, contains setting-agnostic lore involving “Temporal Prime,” posited as “a pseudo-reality that permeates every plane of existence among the alternate realities,” which the authors suggest was previously mis-described in other D&D products as a “Demi-Plane of Time.” (See p. 4).

amazon rds – What Strategies are most advisable for managing users and their privileges in an AWS RDS Database?

Whenever a new database is created, best practices dictate that we should deploy alongside with it several users for several different activities. For example, a typical (at least to me) arrangement would sport at least these 6 database users:

  • an administrative super-user (traditionally, root) will be automatically created by RDS;
  • between one and three users should be created for the application:
    • a read-write,
    • a read-only,
    • a schema-owner user with grants to run DDL statements;
  • a user for monitoring data collection;
  • a role-user that will be used by developers when authenticating through IAM credentials into the database.

As you can see, there is a fair amount of database users that have to be present and managed in the database.

What strategies and best practices would you recommend I use to address user management of my databases across a relatively large fleet of database servers?

JBoss AS v6.1: managing applications

On our installation we typically restart the entire JBoss AS every time we need to restart one of the WARs that are deployed (in expanded form) under …./deploy folder.

I thought I’d be able to individually start/stop/update WARs using the admin console and the CLI, and without stopping the entire AS, but just today I discovered that (a) our WARs are seen as “embedded” and cannot be managed via the console, and (b) there is no CLI for AS, only for EAP.

Can I, perhaps, do the same thing by “brute force”, i.e. removing the expanded WAR from the folder (to effectively undeploy), or replacing it with the new version (to effectively update), while the server runs?

Please advise.

PS. Yes, I know, 6.1 is very, very old, it’s a long story.

amazon rds – What Tools and Strategies are most advisable for managing users and their privileges in an AWS RDS Database?

Whenever a new database is created, best practices dictate that we should deploy alongside with it several users for several different activities. For example, a typical (at least to me) arrangement would sport at least these 6 database users:

  • an administrative super-user (traditionally, root) will be automatically created by RDS;
  • between one and three users should be created for the application:
    • a read-write,
    • a read-only,
    • a schema-owner user with grants to run DDL statements;
  • a user for monitoring data collection;
  • a role-user that will be used by developers when authenticating through IAM credentials into the database.

As you can see, there is a fair amount of database users that have to be present and managed in the database.

Being an adept of DevOps philosophy and methods, I want my database users to be automatically created and maintained. What strategies, best practices, automations, and tools would you recommend I use to address user management of my databases across a relatively large fleet of database servers?

architecture – What kind of software architechture should I use for updating and managing my database product data?

The software

Application 1:
Based on an external API I can retrieve products and put them in my database. Using this API I also update these products (e.g. the price). I once created a script in pure php (no framework or anything, just php) to automaticly update the prices based on the API. This is being executed on a server that I think is not ment for this kind of stuff (SQL Server 2019). Windows task manager is executing it every day at 02:00. When executed it starts a local server (php -S localhost:8000, php’s build-in server). It takes 1/2 minutes if there are no products to updates. If there is one it useally takes 30 minutes – 3 hours. Based on which company updated which products. Some have a million products, some have only 10K.

Application 2:
On the other hand I have created a php script (again pure php) that is acting more like an interface. A user could go there, login to the database and API and add new products. This basicly uses the same code/process as when im updating the products but updating takes some extra checks (e.g. how much a price increased/decreased).

Application 3:
Now I need to create an application that automaticly updates a webshop. Since that database uses a Magento 2 structure I again need slightly different code. But data can be retrieved from the database instead of the API. Should also run every day.

Application 4:
And I need an application where users can define what products should be updated on the webshop.

Application usage

Not many users will use the interfaces, most of it should be set once and then be ok. App1 would probably be used a couple of times a year. App2 runs every day automaticly, no user should touch this. App3 should be used a couple of times per 1 or 2 months and app4 should also run every day without a touch from a user.

Note: I guess I could probably combine app2 and 4 together since there both interfaces. But for now this makes it more clear I guess.

What I want to improve

I belive the security for these php scripts are not at a high level. Also starting a local server on a database server is I think not good. For the interface part of the script the user also has to login to this server before it can start the interface. I also dont like that I have a lot of code that looks very simular and could be copy pasted trough all the 4 applications.

I was thinking of creating an API that manages my database and the webshop its database (and maybe other future databases/tables). Managing code this way seems alot easyer since changes would apply to all applications instead to just one and break the other 3. But I have never created an API, so is an API meant for this kind of stuff? Or should I stick to using specific applications for each script/interface. Maybe im missing something that is better for this.

I also want to rebuild the php scripts into a more secure thing. With Symfony or some other framework.

So, my question: what kind of software architechture should I use for updating and managing my database product data?

RackNerd’s CEO, Dustin B. Cisneros on the Evolution of the Hosting Landscape, Aggressive Expansion, and Managing 17 Datacenter Locations

Dustin Cisneros InterviewRackNerd, a well known provider within the LowEndBox & LowEndTalk community which was recently voted #1 top provider by the community members, was recently featured on an interview with HostAdvice. The editorial team here at LowEndBox has interviewed multiple staff members of RackNerd numerous times in the past on our interview series, but we felt that this interview was also worth sharing considering RackNerd is a popular provider among these parts and we felt like you might enjoy this look as Dustin talks more about the hosting landscape, along with some of his experiences and challenges that he has faced in managing 17 different datacenter locations, while trying to continue on his mission to expand.

Editor’s Note: The opinions and context below are solely between those of the author and interviewer, Dustin B. Cisneros of RackNerd, and Stefanija Kukunovska of HostAdvice respectively. The context mentioned below does not represent the opinions, values, or positions of LowEndBox. LowEndBox is also not affiliated with the interviewer, HostAdvice, in any way. If you or your company would like to take part in a Guest Post, Interview, or Op-Ed, please contact the LowEndBox administration on our Help Desk.

We always start our interviews with some background of our speaker and then a bit more about the company. What is your experience in the hosting industry? When did it all start?

I am Dustin B. Cisneros, and I’m the founder and CEO of RackNerd. To provide some insight into my background — I first entered the hosting industry 13 years ago, with a primary focus on three product lines that I deemed essential: dedicated servers, shared hosting, and reseller hosting. During the early days and stages, I initially ran every department myself — doing so, I learned all the components of the hosting industry and the key importance of customer communication, rapid support, and ensuring that an outstanding hosting experience gets delivered. Fast forward to years later, my company grew to over 4,000 active customers and a dozen employees under my wing.

To deep dive further into the industry, I began working at a data center services company in 2011 that offered, at that time, three US-based facilities. Throughout my tenure, I expanded the company’s footprint to 9 global datacenter locations and through my sales-focused role, I was individually responsible for selling millions of dollars in recurring revenue. This role gave me the opportunity to analyze complex business problems and solve them with elegant and future-focused solutions. While at this organization, I worked my way up the ranks and I was utilizing my experience to lead and manage many of the company’s departments!

After nearly eight years of employment, I desired to leverage my capabilities and expand my personal and professional growth. At this turning point, I resigned from my leadership position and launched RackNerd. Today I serve as the company’s CEO where I showcase the gamut of experiences and best practices I’ve learned — as well as never losing sight of my earlier years where I learned the instrumental importance of delivering customer-focused services.

As of August 2021, the time we are doing this interview, we operate data centers in 17 different locations – serving clients from every continent of the world (except for Antarctica!). RackNerd plans to continue expanding its global footprint, while delivering the highest level of service to our esteemed client base.

When did you found RackNerd? From where did you get the idea and the inspiration for RackNerd?

While RackNerd was launched in 2019, the idea was sparked a number of years prior. While focused upon rounding out my industry and professional experience, I never lost sight of my goal and I would make micro-investments into my future in order to keep the spark alive. I would do little things, such as developing the name and registering it years prior — while focused on expanding my knowledge so that I could launch my company properly. Though the vision has long existed, in 2019 we officially launched and have scaled very quickly since — in what can be attributed to successfully leveraging years of industry experience. From day one, this has ensured customers receive a smooth and seamless experience while we focused on forming a solution-oriented company that serves and fulfills a void in the hosting marketplace. By putting our customers first and by listening to their needs and wants, we have done very well as a company. Not only are we very clear on our mission, our entire team is on the same page of over-delivering and ensuring we are over-exchanging in service and value delivery with our customers.

You have been in the industry for many years now, and we are sure that you have faced ups and downs along the way to success. What were the challenges that you had to face while building RackNerd?

Everyone experiences challenges, both personally and professionally when embarking on launching a new business. One of the challenges we faced (and at times still face) pertains to managing inventory levels in 17 different locations — through aggressive expansion, as well as a myriad of factors, such as client hyper-growth scaling – there are at times, even with modern data analytics, that we sell out available stock before suppliers can replenish our inventory. As demand for our services continues to remain high, a temporary supply chain issue such as this, can have a real adverse impact on any business.

How did you overcome them?

Some might argue that this problem is relatively a good one to have, undeniably it is, but it is one we address on an ongoing basis with the vendors in our supply chain — our goal of course is predictive modeling and having stock ready at time of need. At the time of this discussion, our provisioning team is busy deploying 90 VPS nodes globally, across numerous data center locations to accommodate the run rates we encounter on our VPS-based offerings.

How has the hosting industry changed from the time you enter the world of hosting?

The hosting landscape has changed significantly, and it is constantly evolving. Today we are seeing an increase of XaaS (“delivery of anything” as-a-service) integrated with hosting, and the industry has also experienced leaps and bounds in the sophistication of underlying technology. As an example, OpenVZ and Xen used to be the gold standard in virtualization technology, and now KVM is considered the most desirable among providers and end-users. Industry-wide, more and better options (for example control panels) are emerging, and many previous challenges and prohibitive costs (such as bandwidth pricing) have improved over the years. As a whole, our industry does a great job self-regulating and is extremely resilient as well as innovative!

What are the products and services that RackNerd is offering at the moment?

RackNerd offers shared hosting, reseller hosting, KVM VPS, Windows VPS, Dedicated Servers (both Intel and AMD Ryzen Dedicated Servers), Private Cloud Solutions, Colocation, DDoS Mitigation, Server Management, Fixed Wireless and Managed Services. We offer customized solutions as well for anyone in need of something outside-of-the-box.

How is RackNerd different from the competition?

While fundamentally RackNerd remains competitive in the industry by confronting the challenges each and every business faces, we leverage our industry experience with market intelligence to minimize and eliminate growing pains during our scaling. Our educated and competent staff cares greatly for our clients and the overall sustainable well-being of RackNerd. With a great passion for the industry, we’re taking on the competition, not just on price — but on service.

Customers care about the speed, backup, security of their websites, among other things. How is RackNerd handling customers’ specific needs and requests?

In terms of speed, we believe that a service’s deployment location is key. The general rule of thumb is that the data center location should be closest to the majority of your audience and end-users. This is one reason why we have 17 different data center locations. In addition, we utilize technology such as LiteSpeed Web Server with LSCache to optimize delivery times, and our hardware specifications we put in production are optimized in a manner to deliver the best speeds possible. In terms of backups (which we advocate and educate our clients on the importance of implementing) and matters of security (hardening, options, solutions) — we listen to our clients and we are experienced with asking the right questions to determine how we can better aid their business and objectives. Providing our experience to our clients is one of our primary goals.

As we mentioned before, the industry is constantly changing. What new products, technologies, and trends are you particularly interested in that will be very popular in the following period?

We recently launched our AMD Ryzen and AMD EPYC dedicated servers, as well as Ryzen VPS Hosting based upon underlying AMD Ryzen infrastructure. We have seen demand shift in favor of AMD Ryzen and AMD EPYC CPUs, whereas just a few years ago, AMD was not even considered a contender to Intel in the hosting industry. In this new climate, people are able to rapidly deploy AMD-based servers and receive both stability as well as some of the best performance, for only a fraction of the price compared to an Intel-based equivalent. In addition, we have embraced NVMe technology, which is substantially faster than alternative methods of storage and still very affordable. Many of our VPS offerings are now based on AMD Ryzen and NVMe storage infrastructure, delivering a major improvement over virtualized services (cloud and VPS) that were the only options, not that long ago! 

What are your plans for 2022?

As a globally diversified provider of hosting solutions, we have amassed a portfolio of customers who have location-sensitive (latency) requirements. To achieve our roadmap goals, and with our clients demanding greater global presence, we are actively negotiating contracts to expand our service offerings to additional data center locations. In addition to more automation and streamlining, we want to provide additional geographic options to better cater to client needs.

Where do you see your company in the next ten years? 

We see ourselves continuing to expand based on customer feedback, with more product offerings and additional data center locations. As we’ve learned through experience, by doing so, our market reach and the value we can bring to clients will only expand. Of paramount importance during any planned expansion, we ensure that whenever we scale, we are doing so in such a manner that allows us to maintain our service quality and not experience any “growing pains” that could impact our clients. We have an excellent reputation for unparalleled value delivery and customer support — we intend to keep it that way!

Finally, is there anything else you would like to add about RackNerd that we missed?

We recently expanded to new office space in Southern California, and we’ve accomplished an excessive amount already as a team and have so much more to do. The commitment and passion our team has, without a doubt will help us reach the next level. Thank you for your time today to conduct this interview with me. To anyone reading, I’m always open to connecting with other like-minded individuals or business owners. Feel free to connect with me on LinkedIn, or e-mail me at dustin@racknerd.com if you have any questions (or if you’d just like to say hello!).

raindog308

I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.