Unlock the power of financial data analysis with Python Pandas! Learn essential techniques for inspecting stock market data, understanding data structures, and generating key statistics. #FinancialAnalysis #PythonPandas #DataScience

https://teguhteja.id/mastering-financial-data-analysis-python-pandas/

I need to make a fixed-width text file from a Pandas DataFrame.

I followed a suggestion to use to_string method. I built out my solution only to find it adds a space between the columns in the output. A fixed-width file means each column must be an exact number of characters. I literally can't make the file I need with to_string.

Making a fixed-width text file is a very easy task in SAS. Trying to do this with Pandas has been extremely frustrating.

#PythonPandas #SASProgramming

Spent ages today trying to figure out why the dates in my series weren’t being handled correctly by #PythonPandas to_datetime function.

The default error function for invalid parsing is to raise an exception, and two other values can be set.

The documentation says:

- If 'coerce', then invalid parsing will be set as NaT.
- If 'ignore', then invalid parsing will return the input.

I took ‘ignore’ to mean that an invalid value within a series would be skipped.

It turns out the *entire input series* is skipped, returning the original, unmodified series.

But the *opposite* is true for ‘coerce’, which returns NaT for the invalid value in the series, and processes the valid values.

It felt unintuitive, to say the least (maybe just me?). And I find it odd that there’s not a ‘skip’ option?

Re-reading the docs with the knowledge of what the function actually does, it kinda makes sense? But doesn’t feel sufficiently clear.

Yesterday I learned about, and got to use, one of my favourite #Python #Pandas methods: cumsum.

You cumsum once, and then you cumsum more!

Other top contenders include cumprod, and cummin, but I didn’t get to use those 😂

#PythonPandas

Okay this is kind of absurd but I guess a learning experience. Merging lots of data in #Python using #PythonPandas and a couple hundred frames in, it slows to a crawl. 15+ min and I abort.

Instead do batch processing of a hundred files a piece, then merge the resulting Parquet files again. Less than five minutes.

#TIL