5 Advanced concepts in Python will make your boss happy

Vasanth Jagadeesan

Vasanth Jagadeesan, M Nikhil Swami

— 5 min read
5 Advanced concepts in Python will make your boss happy

Be it work, or personal projects, there are some best practices which must be followed that will differentiate a beginner from Experts. in this article we will explore those tricks and concepts in python which will

  • [x] Save you ton of time
  • [x] Make your boss happy
  • [x] impress your colleagues and juniors.

1. Profiling

"you cant improve what you cant measure" as the saying goes, you need a method to evaluate performance of certain piece of code. the code could be a set of statements, function calls , initiating a class, connecting a database, whatever! there is this amazing built-in python library cProfile that does exactly the job!

import cProfile
import re
cProfile.run('re.compile("foo|bar")')
explanation: we are importing c profile library, in the last statement, we profile the regex re.compile("foo|bar") this finds all occurences of "foo" or "bar" in given piece of text, however we are finding the time required for initiating the compiled function , compiled function can be reused again and again.

Output of profiling

197 function calls (192 primitive calls) in 0.002 seconds
Ordered by: standard name
ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     1    0.000    0.000    0.001    0.001 <string>:1(<module>)
     1    0.000    0.000    0.001    0.001 re.py:212(compile)
     1    0.000    0.000    0.001    0.001 re.py:268(_compile)
     1    0.000    0.000    0.000    0.000 sre_compile.py:172(_compile_charset)
     1    0.000    0.000    0.000    0.000 sre_compile.py:201(_optimize_charset)
     4    0.000    0.000    0.000    0.000 sre_compile.py:25(_identityfunction)
   3/1    0.000    0.000    0.000    0.000 sre_compile.py:33(_compile)

a detailed guide on using this advance technique can be found here The Python Profilers — Python 3.10.7 documentation

PRO TIP: Show this to your boss, that i have improved performance of this function by 235% and consumes X amount of memory! so that you can really demonstrate your effort put in optimizing the codebase.

2. Threading

logic is simple, if one person lifts 1 kg per minute, 2 people lift 2 kg per minute. well its not a perfectly linear ratio in programming but the concept is to distribute compute intensive tasks to a pool of workers so you can parallelize the execution of your code. but you might ask why do i even need parallelizing aka "running concurrently" ? Explanation : imagine a person building a burj-khalifa , the tallest building in world. is it really possible to build such a tall building by one person? NO! the same goes for threading.

Almost all languages have threading with exceptions such as javascript, where there is the concept of workers. in python you can do like this.

import threading

def some_function(arg):
	# lot of code
	print(f"function done : processed arg={arg}") #basic logging f-string used 

t1 = threading.Thread(target=some_function, args=("arg 1",))
t2 = threading.Thread(target=some_function, args=("argument abc",))

# instruct the python runtime to spawn a thread with same environment shared b/w both
t1.start()
t2.start()

# threads need to be joined, to explicitly wait for result
t1.join()
t2.join()
explanation: import the standard threading library, then create a function called some_function . create threads with variable t1 & t2 which can be controlled. the threads have 2 argument/parameters 1st being the function which needs to be parallelized, note: dont give string as input , target="function1234" DONT DO THIS, as putting quotes around a text makes it a string , strings are uncallable and will return error . finally start the threads, and join as per your needs.

OUTPUT:

	function done : processed arg=arg 1
	function done : processed arg=argument abc

As simple as that. more explanation Multithreading in Python 3 - Javatpoint

PRO TIP: its a industry best practice to leverage threading wherever possible, like Database operations, network requests, file handling etc. you can always be proud infront your team that you have used threading. as it differentiates an expert from novices!

3. Regular Expressions

The name is not quite appealing to the audience, i understand! it could have been better named as "pattern matcher" or more simple "textractor" text+extractor. but in python regex is implemented in C, under the hood, it is as fast as it could get. if you were to given a problem like `how many times does "hyderabad occur in this article" the forget splitting lines, then splitting words and check word by word. with regex its just 2 lines.

here is a sample:

import re
line = "hi im Nikhil my email is [email protected]"
match = re.search(r'[\w.+-]+@[\w-]+\.[\w.-]+', line).group(0)
explanation : first line we import regex library. in second line we give a sample string, this could be billions of characters long and your computer wont break a sweat. in third line we use the standard email matching pattern [\w.+-]+@[\w-]+\.[\w.-]+ which extracts emails, the regex matching engine now does rest of the work. you might wonder why such strange characters in this pattern? the answer is regex has its own syntax and its quiet easy to learn.

Output

'[email protected]'

abundant resource is available online, and there is no point putting it all here, however, try full tutorial RegexOne - Learn Regular Expressions STEP BY STEP!

PRO TIP : use it when you want to extract specific patterns like emails or phone numbers etc from gaint corpus of text. or want to filter out negative patterns like foul words or blacklisted words. or just replace person "Adam" with "Betty" any article.

4. Comprehensions

there are 4 types of comprehensions , they are basically replacement for for loops in python. this is because comprehensions iterate over an iterable (string, list, tuple, set) and allow to perform basic operations on element of a list.

  1. List Comprehension : [ len(x) for x in bigList]
    • Calculate length of individual elements in some big list
  2. Dictionary Comprehension : {k:v for k,v in zip(x,y)}
    • create a dictionary by zipping(packing) 2 iterables of equal length
  3. Set Comprehension : { x for x in bigList if x}
    • similar to list comprehension, create a new set, where only elements whose boolean value is truthy, ie exclude 0, False, None, [] empty stuff... that occurs in bigList.
  4. Generator Comprehension: ( len(x) for x in bigList)
    • same as list comprehension but puts out values only when called duing iteration, which saves memory...

PRO TIP : use wherever simple for loop is present in code, it keeps code leaner. dont use if your iteration contains many if-else operation per loop. every line saved in code matters for future maintainability! less is more.

5. Operator Chaining , Short hand If-Else

every code has a lot of if else, statements. and that does increase the length of code. as we know bigger code bases == slower development times, there is this mini trick for short circuiting if-else statements.

simple example focus 3rd line.

givencolor = 'red'
favcolor= 'green'
is_favourite = "yes :)" if givencolor == favcolor else "nope :("
print(f"Nikhil got his favourite color? {is_favourite}")
explanation : in the first 2 lines we assign 2 variables, the names are self-explanatory. in third line we are doing a short circuit to check if given color matches favourite color. in our case "green" and "red" dont match, so is_favourite resolves to false ie the value of else block "nope :(" will be assigned to variable. had given color been "green" then our output would be yes :)

Result

Nikhil got his favourite color? nope :(

Read more