reinforcement learning – Resources on Dynamic Programming with Indefinite Recursion

I am trying to explain the value iteration method that is used in reinforcement learning. The method is used to estimate a solution to a recursive equation like:

$ Return(state_t,action_t) = Reward(state_t,state_{t+1}) + gamma * max begin{bmatrix} Return(state_{t+1}, action_{t+1}==’n’)\ Return(state_{t+1}, action_{t+1}==’e’)\ Return(state_{t+1}, action_{t+1}==’w’)\ Return(state_{t+1}, action_{t+1}==’s’) end{bmatrix}$

Normally, if the sub-problems(RHS) are overlapping, a problem like this can be solved using recursive or iterative dynamic programming. However, in the case of reinforcement learning the recursion is often mutual, and thus the dynamic programming solution has no natural point of termination.

The value iteration method works by filling up the memorization table with arbitrary values and then repeatedly applying the recursive equation to every cell in the table until the values in the cell converge.

A similar approach is used in the HITS algorithm where the hub and authority values have mutual recursion.

Are there any good resources, preferably online, that deal with the application of dynamic programming to problems with mutual, indefinite recursion?

Featured resources | NulledTeam UnderGround


Displays a random featured resource every 10 minutes.

(Example of featured resources widget)



  • Cron Entry will rotate posts every 10 minutes.
  • All phrases start with featuredresources_ for your convenience.


  1. Download and unzip it.
  2. Copy the src/addons/Andy/FeaturedResources directory to your server…

.(tagsToTranslate)nulled(t)nulled sites(t)nulled forum(t)nulled script(t)nulled scripts(t)nulled script forum(t)best nulled scripts site(t)xenforo nulled(t)xenforo 2 nulled(t)xenforo nulled themes(t)seo xenforo 2(t)xenforo themes(t)seo for xenforo(t)vbulletin nulled(t)vbulletin 5 nulled(t)whmcs nulled(t)hexa whmcs(t)whmcs addons nulled(t)whmcs templates nulled(t)whmcs template nulled(t)whmcs modules nulled(t)whmcs themes nulled(t)cs-cart nulled(t)xfilesharing pro nulled(t)blesta nulled(t)arrowchat nulled(t)multi vendor ecommerce script nulled(t)seo providers(t)adsense alternative

online resources – Is there a website listing the location of all the Windows 10 lock screens?

online resources – Is there a website listing the location of all the Windows 10 lock screens? – Travel Stack Exchange

Are long-lived SSL/TLS certificates acceptable for non-public resources?

Our organization maintains a private certificate authority to issue SSL/TLS certificates to non-public backend services running on our cloud infrastructure. For example, Postgres and Elasticsearch servers running on a private vnet.

Currently we issue these certificates with a 2 year expiry, but the renewal process is manual and we don’t have the resources to work on automating it. Given the certificates are only used within a private vnet, are there any significant risks associated with extending the expiry to 5 or 10 years?

Alwyzon: Storage and Performance Servers in Austria with Generous Resources Starting at €2.88/month!

A hosting company named Alwyzon?  I see what you did there…well, at least it’s more clever than pounding WHOIS servers for hours trying to find an unregistered domain in the form of (random noun)  Welcome, Alwyzon!

They’re an Austrian hosting company hosting out of their home country and they’ve got a short-term deal for our reads., including storage servers.  That sounds tasty as I don’t think we’ve seen a storage offer out of Austria before.

Between now and March 24, 2021 at 23:59 CET, you can get:

  • 500G for only €2.88/month if you sign up annually (quarterly also available)
  • 1TB for €4.95/month on annual contract (monthly also available)
  • They’ve also got a 2GB KVM with 100GB of storage and 5TB of bandwidth for only €4.68/month!

These deals use the code HELLOVIENNA to get the discounts shown below.

For one of the more expensive geographies, that’s nice and if you are in that part of Europe and looking for a low-latency connection at a decent price, take a look at Alwyzon.

They are registered as Hohl IT (Austria FN 538746z). Their Terms of Service is available on their web site. They accept Credit Cards, BanContact, EPS, giropay, iDEAL and PayPal.

Here’s a little about Alwyzon in their own words:

“Alwyzon is a trading name of Hohl IT e.U., a small software development firm registered in Graz, Austria. All equipment is fully owned, equity financed and runs on our own network in the InterXion VIE1, one of the largest datacenters in Austria and home to the major Austrian internet hub VIX. As a RIPE NCC member, we also own all IPs assigned to any client and currently operate the equivalent of a /21 IPv4 subnet to offer our services.

Remember that the LEB community would love to hear about your experience in the comments section below!

Now read more to see the offers!

Storage Server

  • 512 MB DDR4 ECC RAM
  • 1x vCPU Xeon® Silver (2nd Gen)
  • 500 GB HDD space (RAID6, Cached, HW)
  • 1 TB Network Traffic included
  • 100 Mbps Bandwidth
  • 1x IPv4
  • /64 IPv6
  • KVM
  • € 8,64/quarter
  • € 34,56/year
  • (ORDER)

Storage Server

  • 1x vCPU Xeon® Silver (2nd Gen)
  • 1 TB HDD space (RAID6, Cached, HW)
  • 2 TB Network Traffic included
  • 1 Gbps Bandwidth
  • 1x IPv4
  • /64 IPv6
  • KVM
  • € 4,95/month
  • € 59,40/year
  • (ORDER)

Virtual Server

  • 1x vCPU Xeon® Silver (2nd Gen)
  • 100 GB SDD Storage (RAID10, HW)
  • 5 TB Network Traffic included
  • 1 Gbps uplink
  • 1x IPv4
  • /64 IPv6
  • KVM
  • € 4,68/month
  • € 56,16/year
  • (ORDER)


  • Datacenter: InterXion VIE1, Vienna, Austria
  • Test IPv4:
  • Test IPv6: 2a0d:f302:100::2
  • Test file:
  • Looking glass:


  • Host node specifications vary depending on the ordered services, but solely consist of the latest generation of Dell PowerEdge servers with 2nd generation Intel Xeon Scalable processors, Hardware RAID controllers (including battery-backed caches for the HDD-based storage servers) and redundant network links.

Please let us know if you have any questions/comments and enjoy!


I’m Andrew, techno polymath and long-time LowEndTalk community Moderator. My technical interests include all things Unix, perl, python, shell scripting, and relational database systems. I enjoy writing technical articles here on LowEndBox to help people get more out of their VPSes.

Similar resources

Admin submitted a new resource:

Similar resources – Shows a list of similar resources when a user is viewing a resource.


Shows a list of similar resources when a user is viewing a resource.

(Example of similar resources)

“lightbox_close”: “Close”,
“lightbox_next”: “Next”,
“lightbox_previous”: “Previous”,
“lightbox_error”: “The requested content cannot be loaded. Please try again later.”,
“lightbox_start_slideshow”: “Start slideshow”,
“lightbox_stop_slideshow”: “Stop slideshow”,
“lightbox_full_screen”: “Full screen”,
“lightbox_thumbnails”: “Thumbnails”,
“lightbox_download”: “Download”,
“lightbox_share”: “Share”,
“lightbox_zoom”: “Zoom”,
“lightbox_new_window”: “New window”,
“lightbox_toggle_sidebar”: “Toggle sidebar”


(Example of options page)


Read more

.(tagsToTranslate)xenforo(t)xenforo themes(t)xenforo nulled(t)xenforo free(t)xenforo 2.2(t)nulled(t)nulled script(t)whmcs(t)whmcs nulled(t)scripts(t)nulled scripts(t)vbulletin(t)vbulletin nulled(t)vbulletin 5 nulled(t)xenforo resources(t)wordpress download(t)wordpress nulled forum

web development – Solution to notify users Single Page Application (SPA) static resources have updated if users don’t refresh their browsers

Basically you want to trigger an event when the version changes to inform the user that their UI is outdated and they should refresh. If you want to keep it simple this can be done file-based. You basically need these components:

  • A JavaScript file (currentversion.js) that dispatches a event about the current version
  • Logic in the SPA that periodically reloads currentversion.js file and defines listeners for the “currentversion” event.

The generation of that currentversion.js and currentVersion can be automatically generated to contain the correct version on every build.

The idea is to (re)load the currentversion.js as needed to dispatch the current version of the UI. The logic can listen for that event and act when the version has deviated from last known version.

In theory you can have something like:


window.dispatchEvent(new CustomEvent('currentversion', { detail: '1.1' }));


            const currentVersion = '1.1';
            function startVersionCheck() {
                const versionJs = document.getElementById('version-js');
                const versionJsUrl = versionJs.src;
                let versionJsReloadCount = 0;
                const versionCheck = function () {
                    versionJs.src = (versionJsUrl, '?rc=', ++versionJsReloadCount).join('');
                setInterval(versionCheck, 60000);

            window.addEventListener('currentversion', function (e) {
                const version = e.detail;
                if (currentVersion !== version) {
                    window.dispatchEvent(new CustomEvent('versionchanged', { detail: version }));

            window.addEventListener('versionchanged', function (e) {
                alert('New UI version available!');

            window.addEventListener('load', startVersionCheck);
        <script id="version-js" src=""></script>

Both the UI and currentversion.js expose the same version initially. When the value changes in currentversion.js it should eventually trigger the versionchanged event on the next reload.

Once the user reloads both version are in sync again.

DreamProxies - Cheapest USA Elite Private Proxies 100 Private Proxies 200 Private Proxies 400 Private Proxies 1000 Private Proxies 2000 Private Proxies - Buy Cheap Private Proxies Buy 50 Private Proxies Buy 100 Private Proxies Buy 200 Private Proxies Buy 500 Private Proxies Buy 1000 Private Proxies Buy 2000 Private Proxies ProxiesLive New Proxy Lists Every Day Proxies123