python 3.x – Improving speed of parsing XML file

import pandas as pd
import pandas_read_xml as pdx
from lxml import etree
url2='http:'
url='http:'
tree=etree.parse(url)
st=etree.tostring(tree.getroot())

# The list "all_leaves" includes every node and leaf in the tree

from io import StringIO, BytesIO
#st above is encoded as binary

all_leaves=()
events = ("start", "end")
# "start" are the xml tags start
# "end" are the xml tags with "</"
parent='root' #root of the xml tree
context = etree.iterparse(BytesIO(st), events=events)
for n, actel in enumerate(context):
  if actel(0)=='end': #this marks the end of a node
    parent = all_leaves(n-1)(1) #use previous parent
    all_leaves.append((actel(0),parent,actel(1).tag))
    #append the action, parent and tag
  else:
    all_leaves.append((actel(0),parent,actel(1).tag))
    if actel(0)=='start': #the current become parent since this
                          #algorithm is depth first
      parent = actel(1).tag

# "leaves" includes only the first row of each table
leaves=()
for entry in all_leaves:
  if (entry not in leaves) and (entry(0)=='start'):
    leaves.append(entry)


# These are the names of the tables
nf_nodes=()
for entry in leaves(2:):#ignore the first two:
  nf_nodes.append(entry(1))
nf_nodes=list(set(nf_nodes))


# The dictionary "all_dic" has keys labeled as the node
# above each set of leaves (i.e. DataFrames) with value equal to the corresponding
# DataFrame or leav
all_dic={}
for l in nf_nodes:
    all_dic(l)=pdx.read_xml(url, ('DRFAssessments',l))

I have the following code below that reads and parses xml files. The following code takes about 10:30 seconds to run. I wanted to see if there were any suggestions or any edits that could be made to the code to increase the speed. The code is used to parse the xml file before it is uploaded to a MYSQL database. It also parses 20 tables if that is any useful

sql server – Speed up Key Lookup (clustered)

I have a slow query in Sql 2017. Key lookup (Clustered) is taking 62%. There is also an Index Seek (nonclustered) taking 30%.

Is there anyway I can speed this query up?

Execution plan: https://www.brentozar.com/pastetheplan/?id=r1oIcWofd

exec sp_executesql 
N'
SELECT TOP(@__p_4) 
    (v).(County), 
    (v).(Community), 
    (v).(DataProvider), 
    (v).(Site), 
    (v).(Latitude), 
    (v).(Longitude), 
    (v).(Elevation), 
    (v).(Instrument), 
    (v).(MonitorId), 
    (v).(ParameterId), 
    (v).(Parameter), 
    (v).(MeasurementStartTime), 
    (v).(MeasuredValue), 
    (v).(AdjustedValue), 
    (v).(Units), 
    (v).(AdjustmentDesc), 
    (v).(LowerLimitCheck), 
    (v).(UpperLimitCheck), 
    (v).(RepeatingValueCheck), 
    (v).(SpikeCheck), 
    (v).(QualityPreliminary)
FROM (vw_DownloadDataSubhourly) AS (v)
WHERE 
(
    (((v).(CountyId) = @__Parse_0) AND ((v).(MeasurementStartTime) >= @__start_1)) 
    AND ((v).(MeasurementStartTime) <= @__end_2)
) 
AND (v).(ParameterType) IN (''PM2.5'')
ORDER BY (v).(Site), (v).(MonitorId), (v).(ParameterId), (v).(MeasurementStartTime)
',
N'@__p_4 int,@__Parse_0 int,@__start_1 datetime,@__end_2 datetime',
@__p_4=750000,@__Parse_0=34,@__start_1='2019-10-01 00:00:00',@__end_2='2020-12-31 23:59:59'

vw_DownloadDataSubhourly
Execution plan: https://www.brentozar.com/PasteThePlan/?id=BkiX1Vsz_


CREATE view (dbo).(vw_DownloadDataSubhourly) as

SELECT      
                DataProvider.DataProviderId, 
                County.CountyId, 
                Community.CommunityId, 
                County.CountyName as "County",
                Community.CommunityName as "Community",
                DataProvider.DataProviderShortName as "DataProvider", 
                Site.SiteLongName as "Site",
                SiteConfig.LatitudeTrunc as "Latitude",
                SiteConfig.LongitudeTrunc as "Longitude",
                SiteConfig.Elevation,
                Monitor.ExternalMonitorId as "MonitorId",
                Parameter.ParameterId as "ParameterId",
                Parameter.ParameterNamePublic "Parameter",
                Parameter.ParameterType "ParameterType",
                SubHourlyMeasurementData.StartDateTime "MeasurementStartTime",
                SubHourlyMeasurementData.AObs as "MeasuredValue",
                SubHourlyMeasurementData.AObsAdj as "AdjustedValue",
                UOM.UOMAbbreviation as "Units",
                AdjCode.AdjCodeDescription as "AdjustmentDesc",
                concat(MonitorConfig.MonitorMfr, ' ', MonitorConfig.MonitorModel) as "Instrument",
                case 
                    when SubHourlyMeasurementData.QF06 = '-3' then 'QC not computed'
                    when SubHourlyMeasurementData.QF06 = '-2' then 'Not enough data'
                    when SubHourlyMeasurementData.QF06 = '-1' then 'Not enough site/monitor info'
                    when SubHourlyMeasurementData.QF06 = '0' then 'Pass'
                    when SubHourlyMeasurementData.QF06 = '1' then 'Fail'
                    else NULL
                    end as "LowerLimitCheck",
                case 
                    when SubHourlyMeasurementData.QF05 = '-3' then 'QC not computed'
                    when SubHourlyMeasurementData.QF05 = '-2' then 'Not enough data'
                    when SubHourlyMeasurementData.QF05 = '-1' then 'Not enough site/monitor info'
                    when SubHourlyMeasurementData.QF05 = '0' then 'Pass'
                    when SubHourlyMeasurementData.QF05 = '1' then 'Fail'
                    else NULL
                    end as "UpperLimitCheck",
                case 
                    when SubHourlyMeasurementData.QF04 = '-3' then 'QC not computed'
                    when SubHourlyMeasurementData.QF04 = '-2' then 'Not enough data'
                    when SubHourlyMeasurementData.QF04 = '-1' then 'Not enough site/monitor info'
                    when SubHourlyMeasurementData.QF04 = '0' then 'Pass'
                    when SubHourlyMeasurementData.QF04 = '1' then 'Fail'
                    else NULL
                    end as "RepeatingValueCheck",
                case 
                    when SubHourlyMeasurementData.QF07 = '-3' then 'QC not computed'
                    when SubHourlyMeasurementData.QF07 = '-2' then 'Not enough data'
                    when SubHourlyMeasurementData.QF07 = '-1' then 'Not enough site/monitor info'
                    when SubHourlyMeasurementData.QF07 = '0' then 'Pass'
                    when SubHourlyMeasurementData.QF07 = '1' then 'Fail'
                    else NULL
                    end as "SpikeCheck",
                case 
                    when SubHourlyMeasurementData.QOverall = '-3' then 'QC not computed'
                    when SubHourlyMeasurementData.QOverall = '-2' then 'Not enough data'
                    when SubHourlyMeasurementData.QOverall = '-1' then 'Not enough site/monitor info'
                    when SubHourlyMeasurementData.QOverall = '0' then 'Passed all checks'
                    when SubHourlyMeasurementData.QOverall = '1' then 'Suspect'
                    else NULL
                    end as "QualityPreliminary"

FROM           SubHourlyMeasurementData 
                         INNER JOIN
                         DataProvider ON SubHourlyMeasurementData.DataProviderId = DataProvider.DataProviderId 
                         INNER JOIN
                         Site ON SubHourlyMeasurementData.SiteId = Site.SiteId AND Site.DataProviderId = DataProvider.DataProviderId
                         LEFT JOIN Community ON Site.CommunityId = Community.CommunityId
                         INNER JOIN 
                         SiteConfig ON SiteConfig.SiteId = SubHourlyMeasurementData.SiteId
                         INNER JOIN
                         County ON County.CountyId = SiteConfig.CountyId
                         INNER JOIN
                         Monitor ON Monitor.MonitorId = SubHourlyMeasurementData.MonitorId
                         INNER JOIN
                         MonitorConfig ON MonitorConfig.MonitorConfigId = SubHourlyMeasurementData.MonitorConfigId
                         INNER JOIN
                         Parameter ON Parameter.ParameterId = SubHourlyMeasurementData.ParameterId
                         INNER JOIN
                         UOM ON SubHourlyMeasurementData.UOMId = UOM.UOMId
                         LEFT JOIN
                         AdjCode ON SubHourlyMeasurementData.AdjCodeId = AdjCode.AdjCodeId
                         INNER JOIN
                         StreamSegment ON SubHourlyMeasurementData.StreamSegmentId = StreamSegment.StreamSegmentId
                         INNER JOIN 
                         MeasurementTechnique ON MeasurementTechnique.MeasurementTechniqueId = StreamSegment.MeasurementTechniqueId
                         AND MeasurementTechnique.ParameterId = SubHourlyMeasurementData.ParameterId
                         INNER JOIN
                         SamplingDuration ON SamplingDuration.SamplingDurationId = StreamSegment.SamplingDurationId
                         INNER JOIN
                         SamplingFrequencyUnits ON SamplingFrequencyUnits.SamplingFrequencyUnitsId = StreamSegment.SamplingFrequencyUnitsId

WHERE SubHourlyMeasurementData.StartDateTime >= MonitorConfig.StartDateTime and StreamSegment.IsDownloadable = 1 and SubHourlyMeasurementData.iscalculated = 1 

GO

Is there a need in Magento 2.4 for Cache warmer extension or Google Page Speed Optimizer?

Magento ver. 2.4.2
4 x 2.50 GHz Cores
8 GB RAM
5 TB Bandwidth
5000 Mbps Network Out

We used to run Magento 1.9.4 but are switching to Magento 2.4.2. Every time I installed Magento 2.x it always seemed very slow on the back-end (we haven’t set up the front end yet to see) even though I have done all the recommended optimisations.

Now I found 2 extensions created by Amasty:

  • Full Page Cache Warmer
  • Google Page Speed Optimizer

They are not the cheapest ones so I was wondering if this would help with Magento 2 or is Magento 2.x designed in such a way where this is no longer needed? Historically (on our Magento 1.9.4) we had a hard time with Caching extensions so we ended up disabling it (I guess we weren’t able to configure them properly).

python – Improve speed of my code in competitive programming

I’ve just started doing programming problems.

https://icpcarchive.ecs.baylor.edu/external/58/p5880.pdf

The code below works fine, but my run time is 2.715 seconds and the time limit is 3 seconds.

How can I improve speed of this code?

I’ve already started to use sys.stdin instead of input().

import string
import sys


while True:
    key = sys.stdin.readline().strip()
    key = int(key) if key.isdigit() else key
    if key == 0: break

    decrypted_msg = sys.stdin.readline().strip()
    decrypted_msg = int(decrypted_msg) if decrypted_msg.isdigit() else decrypted_msg
    if decrypted_msg == 0: break

    key = (len(decrypted_msg)//len(key) + 1) * key
    key = key(0:len(decrypted_msg))

    encrypted_msg = ""

    for i in range(len(decrypted_msg)):
        key_idx = string.ascii_uppercase.index(key(i))
        deciphered_idx = string.ascii_uppercase.index(decrypted_msg(i))

        enciphered_idx = (deciphered_idx + key_idx + 1) % len(string.ascii_uppercase)
        encrypted_msg += string.ascii_uppercase(enciphered_idx)

    print(encrypted_msg)

python – How to speed up the code for LeetCode “Container with most water” task?

I’m trying to solve the LeetCode question where you need to find out the area of the container with the most water. I have created the solution that seems to work in my testing but it fails for being too slow when I try to submit it on LeetCode.

My idea is that I create a dictionary of (x, y) tuples from the input list and then for every item I need to find the maximum distance to any of other lines that are equal or taller than it and from here I can calculate what the maximum area is possible for this line.

How else can I approach this to get it to run faster? (I can’t submit solution successfully so can’t see examples of answers by other users)

def max_area(height) -> int:
    areas = ()
    coords = {x: (x, y) for x, y in enumerate(height)}
    for x in coords:
        higher = (k for k in coords if coords(k)(1) >= coords(x)(1))
        area = max(abs(coords(j)(0) - coords(x)(0)) for j in higher) * coords(x)(1)
        areas.append(area)
    return max(areas)

Host.co.in – 25% OFF Cloud Hosting | DDos Protected | High Speed SSD | 24/7 Support | Proxies-free

Host.co.in was founded in 2005 and offered High Quality, Reliable, and Affordable Web Hosting Services in India. We are the fast-growing and wildly successful web hosting company focused on delivering high performance, secure IT solutions and services.

For those webmasters, who are looking for Cheap Web Hosting, Host.co.in is the Best Solution. With Host.co.in, stand apart in the sphere of World Wide Web with hosting for all your business needs enabling you to undertake smooth execution of your core business.

25% OFF on Cloud Hosting Plans, Use Code 25CH Offer ends on 28’Feb 2021

Below is the list of Cloud Hosting plans:

Cloud Server Starter

vCPU – 2 vCPU
Memory – 2 GB
Primary Disk – 50 GB
Bandwidth – 200 GB
OS – CentOS 7
Offsite Backup – 100 GB
Firewall (Optional)

>>>> Price ₹5,175/mo With Code 25CH | Buy Now

Cloud Server Basic

vCPU – 4 vCPU
Memory – 4 GB
Primary Disk – 100 GB
Bandwidth – 400 GB
OS – CentOS 7
Offsite Backup – 100 GB
Firewall (Optional)

>>>> Price ₹6,975/mo With Code 25CH | Buy Now

Cloud Server Premium

vCPU – 6 vCPU
Memory – 4 GB
Primary Disk – 150 GB
Bandwidth – 600 GB
OS – CentOS 7
Offsite Backup – 200 GB
Firewall (Optional)

>>>> Price ₹8,775/mo With Code 25CH | Buy Now

Cloud Server Advanced

vCPU – 8 vCPU
Memory – 6 GB
Primary Disk – 200 GB
Bandwidth – 800 GB
OS – CentOS 7
Offsite Backup – 300 GB
Firewall (Optional)

>>>> Price ₹10,950/mo With Code 25CH | Buy Now

For a full list of Cloud Server features, Visit:


Why CHOOSE US?

• Complete Control
• Host multiple websites
• Strong Platform
• Private Server Hosting
• Exceptional Performance
• Managed Servers
• Optional Control Panel
• 24×7 Live Support

Contact information:

In case you have any questions, you can contact our sales department by initiating a chat or by dropping an email to sales@host.co.in or call us on 1800 212 6630.

pandas – Python speed up dataframe writes

Sorry, I’m very new to python. I’m aware that iterating frames in pandas is a very slow operation. I’ve heard vectorization speeds things up. I found a project that I’m using for sentiment analysis on reddit.

The Calculate dataframe method wasn’t working for some reason, (maybe different version of panda) I wanted to refactor it to use vectorization but Sadly it took me 2 days to even get the code working.

data_df = df.filter(('tickers', 'score', 'sentiment'))

tickers_processed = pd.DataFrame(df.tickers.explode().value_counts())

tickers_processed = tickers_processed.rename(columns = {'tickers':'counts'})

tickers_processed('score') = 0.0
tickers_processed('sentiment') = 0.0

for idx, row_tick in enumerate(tickers_processed.itertuples(), 1):
    upd_row_sent = row_tick.sentiment
    upd_row_score = row_tick.score
    for row_data in data_df.itertuples():
        if(row_tick.Index in row_data.tickers):
            upd_row_sent += row_data.sentiment('compound') / row_tick.counts
            upd_row_score += int(row_data.score) / row_tick.counts
            tickers_processed.set_value(row_tick.Index, 'sentiment', upd_row_sent)
            tickers_processed.set_value(row_tick.Index, 'score', upd_row_score)

Calling set value twice seems redundant, If I could calculate the dataset and update it using vectors I think the code would be alot nicer

dnd 5e – Can I use Zephyr Strike outside of combat to increase my running speed?

The spell Zephyr Strike, from XGE, has the following text:

You move like the wind. Until the spell ends, your movement doesn’t provoke opportunity attacks. Once before the spell ends, you can give yourself advantage on one weapon attack roll on your turn. That attack deals an extra 1d8 force damage on a hit. Whether you hit or miss, your walking speed increases by 30 feet until the end of that turn.

Clearly this spell was written with combat in mind. I’m curious what support, if any, exists for using it outside of combat for the purpose of increasing a character’s movement speed, for instance in a foot chase.

What I would like to happen:
I cast Zephyr Strike, waive the attack, and immediately trigger the bonus 30 feet of speed (say, to 60 ft). Since Zephyr Strike’s casting time is a bonus action, I then take the dash action, doubling my speed to 120 feet. I have this speed for 6 seconds.

My interpretation of the spell as written:
The bonus to speed requires the player to take the attack action. The extra speed only lasts until the end of the turn. So to gain the speed benefit, a player must attack something, which uses their action and prevents the use of the dash action while under the effect of the spell.

Obviously the DM can rule however they want, but I’m curious what support the rules might have for my preferred outcome.