Python

Checking if all the values are same in Numpy Array (both 1D and 2D) :

In this article we will discuss how we can check if all elements are same in 1D or 2D. We will also check if all rows or columns have same values or not.

Check if all elements are equal in a 1D Numpy Array using numpy.all() :

Here we compare all the elements with the first element of the array and returns a bool array of same size. If first element is equal to a element in main array, then true is returned else false is returned.

If all elements in bool array become true then all elements of main array are same.

import numpy as sc

# creating a 1D numpy array

num_arr = sc.array([14, 14, 14, 14, 14, 14])

# here we check if all values in array is same as first element

res = sc.all(num_arr == num_arr[0])

if res:

 print('All elements are same')

else:

 print('All elements are not same')

Output :
All elements are same

Check if all elements are equal in a 1D Numpy Array using min() & max() :

If maximum value of element is equal to minimum value of element in array then it indirectly means all values are same in the array.

import numpy as sc

# creating a 1D numpy array

num_arr = sc.array([14, 14, 14, 14, 14, 14])

# if min element= max element then all values in array are same

res = sc.max(num_arr) == sc.min(num_arr)

if res:

 print('All elements are equal')

else:

 print('All elements are not equal')

Output :
All elements are equal

Check if all elements are equal in a Multidimensional Numpy Array or Matrix :

We know that numpy.ravel() function returns flattened 1D view of a array. So we can easily convert a multi-dimensional flattened to 1D array and then compare first element with all the elements to check all element are same or not.

import numpy as sc

mul_arr = sc.array([[1, 1, 1],

[1, 1, 1],

[1, 1, 1]])

# to get a flattened 1D view of multidimensional numpy array

flatn_arr = sc.ravel(mul_arr)

# to check if all value in multidimensional array are equal

res = sc.all(mul_arr==flatn_arr[0])

if res:

 print('All elements are same')

else:

 print('All elements are not same')

Output :
All elements are same

Find rows or columns with same values in a matrix or 2D Numpy array :

Find rows with same values in a matrix or 2D Numpy array :

Similarly to check if all elements are same in each row, we can compare elements of each row with first element.

import numpy as sc
mul_arr = sc.array([[1, 1, 1],
[1, 21, 1],
[1, 1, 1]])
# to compare 1st element with elements of each row
for i in range(mul_arr.shape[0]):
 if sc.all(mul_arr[i]==mul_arr[i][0]):
  print('Row:', i)
print("Following rows have same elements")

Output :
Row: 0
Row: 2
Following rows have same elements

Find columns with same values in a matrix or 2D Numpy array :

Here also to check if all elements are same in each column, we can compare elements of each column with first element.

import numpy as sc
mul_arr = sc.array([[1, 1, 1],
[1, 21, 1],
[1, 1, 1]])
#to compare 1st element with elements of each row
trans_Arra = mul_arr.T
for i in range(trans_Arra.shape[0]):
 if sc.all(trans_Arra[i] == trans_Arra[i][0]):
  print('Column: ', i)
print("Following rows have same elements")

Output :
Column:  0
Column:  2
Following rows have same elements

Reversing a 1D and 2D numpy array using np.flip() and [] operator in Python.

Reverse 1D Numpy array using ‘[]’ operator :

By not passing any start or end parameter, so by default complete array is picked. And as step size is -1, so elements selected from last to first.

# Program :

import numpy as sc

# Creating a numpy array
num_arr = sc.array([11,22,33,44,55,66])

print('Original Array: ',num_arr)

# To get reverse of numpy array
rev_arr = num_arr[::-1]

print('Reversed Array : ', rev_arr)

Output :
Original Array:  [11 22 33 44 55 66]                                
Reversed Array :  [66 55 44 33 22 11]

Reverse Array is View Only :

Here if we do any modification in reversed array, it will also be reflected in original array.

# Program :

import numpy as sc

# Creatin a numpy array
num_arr = sc.array([11,22,33,44,55,66])

# To get reverse of numpy array
rev_arr = num_arr[::-1]
rev_arr[4]=63

print('Modified reversed Array : ', rev_arr)
print('Original array is: ',num_arr)

Output :
Modified reversed Array :  [66 55 44 33 63 11]                      
Original array is:  [11 63 33 44 55 66]

Reverse Numpy array using np.flip() :

flip() function provided by Python’s numpy module helps to flip or reverse the content of numpy array.

Syntax : numpy.flip(arr, axis=None)

where,

• arr :  A numpy array
• axis : Axis along which contents need to be flipped. If None, contents will be flipped along axis of array.

Reverse 1D Numpy array using np.flip() :

Here as it is 1 -D Numpy array, there is no need of axis parameter.

# Program :

import numpy as sc

# To create a Numpy array
num_arr = sc.array([11,22,33,44,55])

print('Original array: ',num_arr)

# To get reverse of numpy array
rev_arr = sc.flip(num_arr)

print('Reversed Array is : ', rev_arr)

Output :
Original array:  [11 22 33 44 55]
Reversed Array is :  [55 44 33 22 11]

Reverse 2D Numpy Array using np.flip() :

Reverse contents in all rows and all columns of 2D Numpy Array :

Here we don’t provide parameter in np.flip() function, then contents will be reversed along the axes of 2-D Numpy array.

# Program :

import numpy as sc

# to create a 2D Numpy array
twoD_Arr = sc.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

print('Original Array is: ',twoD_Arr)

# to reverse 2D numpy array
rev_Arr = sc.flip(twoD_Arr)

print('Reversed Array : ')
print(rev_Arr)

Output :
Original Array is:
[[1 2 3]
[4 5 6]
[7 8 9]]
Reversed Array :
[[9 8 7]
[6 5 4]
[3 2 1]]

Reverse contents of all rows only in 2D Numpy Array :

If we provide axix parameter i.e. axis=1, then rows of array will be reversed.

# Program :

import numpy as sc

# to create a 2D Numpy array
twoD_Arr = sc.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

print('Original Array is: ',twoD_Arr)

# to reverse the content of each row in array
rev_Arr = sc.flip(twoD_Arr, axis=1)

print('Reversed Array : ')
print(rev_Arr)

Output :
Original Array is:  [[1 2 3]
[4 5 6]
[7 8 9]]
Reversed Array :
[[3 2 1]
[6 5 4]
[9 8 7]]

Reverse contents of all columns only in 2D Numpy Array :

If we provide axix parameter i.e. axis=0, then rows of array will be reversed.

# Program :

import numpy as sc

# to create a 2D Numpy array
twoD_Arr = sc.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

print('Original Array is: ',twoD_Arr)

# to reverse the content of each column in array
rev_Arr = sc.flip(twoD_Arr, axis=0)

print('Reversed Array : ')
print(rev_Arr)

Output :
Original Array is:
[[1 2 3]
[4 5 6]
[7 8 9]]
Reversed Array :
[[7 8 9]
[4 5 6]
[1 2 3]]

Reverse contents of only one row in 2D Numpy Array :

Let, we want to reverse only 1st row of a Numpy array.

# Program :

import numpy as sc

# to create a 2D Numpy array
twoD_Arr = sc.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

print('Original Array is: ',twoD_Arr)

# to reverse only 1st row
twoD_Arr[0] = sc.flip(twoD_Arr[0])

print('Reversed Array : ')
print(twoD_Arr)

Output :
Original Array is:
[[1 2 3]
[4 5 6]
[7 8 9]]
Reversed Array :
[[3 2 1]
[4 5 6]
[7 8 9]]

Reverse contents of only one column in 2D Numpy Array :

Let, we want to reverse only 3rd column of a Numpy array.

# Program :

import numpy as sc

# to create a 2D Numpy array
twoD_Arr = sc.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

print('Original Array is: ',twoD_Arr)

# to reverse the content of 3rd column in array
twoD_Arr[:,2] = sc.flip(twoD_Arr[:,2])

print('Reversed Array : ')
print(twoD_Arr)

Output :
Original Array is:
[[1 2 3]
[4 5 6]
[7 8 9]]
Reversed Array :
[[1 2 9]
[4 5 6]
[7 8 3]]

How to move files and directories in python.

In python, shutil module offers various files related operations

Syntax- shutil.move(src, dst)

It accepts source and destination path and moves file or directory from source pointed as src to destination pointed as dst.

Move a file to an another directory :

 We will pass source file in first parameter and destination file in second parameter as string. Some points to keep in view.

• If there is no file in destination directory then a new file will be created with the same name.
• If already exists a file with same name in destination directory, then the file will be overwritten.
• If in destination path, the path is not valid not existed it will result in FileNotFoundError.

import shutil
cre_path = shutil.move('satya.txt', 'document')    
print(cre_path)

Output :
FileNotFoundError: [Errno 2] No such file or directory: 'saya.txt'

Move a file with a new name :

 If we pass a new name of file in destination path, it will reflect in source file as it will move to source file with new name.

• If there was already existed file with same name, then the file will be overwritten.
• If path doesn’t exist for destination path it will result in error.

import shutil
cre_path = shutil.move('satya.txt', 'document/sample.txt')
print(cre_path)

Output :
FileNotFoundError: [Errno 2] No such file or directory: 'satya.txt'

Move all files in a directory to an another directory recursively :

Let there is a situation where we want move all the files form one directory to another directory. To implement this, using shtil.move(), we will iterate all files in source directory and move each file to destination directory.

import shutil,os,glob
def mov_every_Files_Dir(sourDir, destDir):
    print(sourDir)
    print(destDir)
    # checking if both the are directories are not
    if os.path.isdir(sourDir) and os.path.isdir(destDir) :
        # Iterate through all the files in source directory
        for filePath in glob.glob(sourDir + '/*'):
            # moving the files to destination directory
            print(file_path)
            shutil.move(file_path, destDir);
    else:
        print("srcDir & dstDir should be Directories")   
def main():        
    if __name__ == '__main__':
        main()
    srcDir = '/users/sjones/chem/notes'
    desDir =  '/users/sjones/chem/notes_backup'
    mov_every_Files_Dir(srcDir,desDir)

Move file and Create Intermediate directories :

                We know that if there is no directory in a given path, then shutil.move() will give error. So we will create a function which move the file to destination directory and also create all directories in given path.

import shutil, os, glob
def moven_cre_Dir(srcPath, destDir):
    if os.path.isdir(destDir) == False:
        os.makedirs(destDir); 
    shutil.move(srcPath, destDir);
def main():
    if __name__ == '__main__':
        main()
    moven_cre_Dir(srcDir, destDir)
    srcFile = 'certificate/document.txt'
    destDir =  'certificate/document9'

Move symbolic links :

If source file is a symbolic link, then a link will be created at destination path that will point to source link and subsequently source link will be deleted.

Move a directory to an another directory :

We can move an entire directory to other locations by keeping a view on some points.

import shutil
sour_dir = 'satya'
dest_dir =  'satya1'
shutil.move(sour_dir, dest_dir)

Output :
FileNotFoundError: [Errno 2] No such file or directory: 'satya'

• If there was already a destination directory, then source directory will move to destination directory.
• If destination directory is not there, then it will be created.
• If there is no intermediate directory or path is invalid then there will be error.
• If there is another directory with same name as that of the source in the destination directory, then there also will be error.

How to delete specific lines in a file in a memory-efficient way ?

In this article, we will see how to delete a set of lines from a file in various ways.

We can not delete the lines from the file directly so we will first create a temporary file and write into it all the other lines. Then we will delete the original file and rename the temporary file.

We will be using the following file for demonstration

 File.txt

Line 1

Line 2

Line 3

Line 4

Line 5

Delete a line from a file by specific line number in python :

The algorithm will be-

• Open the original file in reading mode
• Enter the line number
• Create a new temporary file and opening it in write mode
• Read the contents of the file while keeping the count of the lines
  1. If the counter reaches the line we have to delete then skip this line
• If any line was removed from the original file then delta the original file and rename the temporary file to that of the original file.
• Else delete the temporary file.

import os
def deleteLine(originFile, lineNo):
    isSkipped = False
    index = 0
    tempFile = originFile + '.bak'
    #Open original file in read and temp file in write mode
    with open(originFile, 'r') as readObj, open(tempFile, 'w') as writeObj:
        #Copy all dat line by line
        for line in readObj:
            #When the loop counter is same as the line number skip
            if index != lineNo:
                writeObj.write(line)
            else:
                isSkipped = True
            index += 1
    #If any lines are not there rename temp as original file
    if isSkipped:
        os.remove(originFile)
        os.rename(tempFile, originFile)
    else:
        os.remove(tempFile)

#Line numbering starts from 0
deleteLine('file.txt',1)

After execution,

File.txt

Line 1

Line 3

Line 4

Line 5

Delete multiple lines from a file by line numbers :

The algorithm will be-

• Open the original file in reading mode
• Enter the line numbers to delete and pass it as a series
• Create a new temporary file and opening it in write mode
• Read the contents of the file while keeping the count of the lines
  1. If the counter reaches the numbers we have, then skip
• If any line was removed from the original file then delta the original file and rename the temporary file to that of the original file.
• Else delete the temporary file.

import os
def deleteLine(originFile, lineNo):
    isSkipped = False
    index = 0
    tempFile = originFile + '.bak'
    #Open original file in read and temp file in write mode
    with open(originFile, 'r') as readObj, open(tempFile, 'w') as writeObj:
        #Copy all dat line by line
        for line in readObj:
            #When the loop counter is same as the line number skip
            if index not in lineNo:
                writeObj.write(line)
            else:
                isSkipped = True
            index += 1
    #If any lines are not there rename temp as original file
    if isSkipped:
        os.remove(originFile)
        os.rename(tempFile, originFile)
    else:
        os.remove(tempFile)

#Line numbering starts from 0
deleteLine('file.txt',[0,2,3]

After execution,

file.txt

Line 2

Line 5 

Delete a specific line from the file by matching content :

The algorithm will be-

• Open the original file in reading mode
• Create a temporary file
• Copy all contents from original file to the temp file line by line. If the line matches the lines we want to delete then skip
• Compare both the files, if there are any difference delete original file and rename temp file as original.

import os
def deleteLine(originFile, lineToDelete):
    isSkipped = False
    tempFile = originFile + '.bak'
    #Open original file in read and temp file in write mode
    with open(originFile, 'r') as readObj, open(tempFile, 'w') as writeObj:
        #Copy all data line by line
        for line in readObj:
            currentLine = line
            if line[-1] == '\n':
                currentLine = line[:-1]
            # if currentLine matches with the given line then skip
            if currentLine != lineToDelete:
                writeObj.write(line)
    #If any lines are not there rename temp as original file
    if isSkipped:
        os.remove(originFile)
        os.rename(tempFile, originFile)
    else:
        os.remove(tempFile)

#Line numbering starts from 0
deleteLine('file.txt','Line 4')

After execution,

file.txt

Line 1

Line 2

Line 3

Line 5

Delete specific lines from a file that matches the given conditions :

The algorithm will be-

• Accept the original file with a function as call-back.
• Open the original file in reading mode
• Enter the line numbers to delete and pass it as a series
• Create a new temporary file and opening it in write mode
• Read the contents of the file while keeping the count of the lines
  1. Pass each line into the function, if it returns true then skip
• If any line was removed from the original file then deleted the original file and rename the temporary file to that of the original file.
• Else delete the temporary file.

import os
def deleteLine(originFile, conditionalFunc):
    isSkipped = False
    tempFile = originFile + '.bak'
    #Open original file in read and temp file in write mode
    with open(originFile, 'r') as readObj, open(tempFile, 'w') as writeObj:
        #Copy all data line by line
        for line in readObj:
#Chech each file by passing it into the function
            if conditionalFunc(line) == False:
                writeObj.write(line)
    #If any lines are not there rename temp as original file
    if isSkipped:
        os.remove(originFile)
        os.rename(tempFile, originFile)
    else:
        os.remove(tempFile)

#Line numbering starts from 0
deleteLine('file.txt',conditionalFunction)

We can pass any function to check for our condition and when they are met, the lines are skipped.. The conditions can be anything like size of the line, line with a particular word in it etc.

String to datetime or date object in Python.

We will see in this article how to convert different format string to a DateTime or date object. We will do so using a function provided by Python’s date-time module to change string into a datetime object i.e. strptime( )

Syntax-

datetime.strptime(datetime_str, format):

Arguments:

• datetime_str – It takes the date and time information
• format – The format in which we want the date item information based on which the object will be created

The function will return the  date time object when the right parameters are provided. In case there is something wrong with the parameters provided to the function will throw a value error.

Convert string (‘DD/MM/YY HH:MM:SS ‘) to datetime object :

We can obtain the date time in our desired format by passing in the correct format code as arguments.

• %d – Days
• %m – Month
• %y – Years
• %H – Hours
• %M – Minutes
• &S – Seconds
• %z – Timezone
• %f – Milliseconds

from datetime import datetime
dateTimeString = '15/5/21 11:12:13'
# Converting the String ( ‘DD/MM/YY HH:MM:SS ‘) into a datetime object
dateTimeObj = datetime.strptime(dateTimeString, '%d/%m/%y %H:%M:%S')
print(dateTimeObj)

Output :
2021-05-15 11:12:13

Convert string (‘MM/DD/YY HH:MM:SS ‘) to datetime object :

To achieve the ‘MM/DD/YY HH:MM:SS ‘ format we are goin to change the order of format from the previous code.

from datetime import datetime
dateTimeString = '5/15/2021 11:12:13'
#Converting the String ( ‘MM/DD/YY HH:MM:SS‘) into a datetime object
dateTimeObj = datetime.strptime(dateTimeString, '%m/%d/%Y %H:%M:%S')
print(dateTimeObj)

Output :
2021-05-15 11:12:13

Convert string to datetime and handle ValueError :

In case we pass a format that is not compatible with the with the function, it throws a ValueError.

We can handle that error beforehand so that the program execution does not stops.

(For correct format)

from datetime import datetime
dateTimeString = '5/15/2021 11:12:13'
#Converting the String ( ‘MM/DD/YY HH:MM:SS‘) into a datetime object
try:
    dateTimeObj = datetime.strptime(dateTimeString, '%m/%d/%Y %H:%M:%S')
    print(dateTimeObj)
except ValueError as e:
    print(e)

Output :
2021-05-15 11:12:13

(For wrong format)

from datetime import datetime
dateTimeString = '5/15/2021 11:12:13'
#Converting the String ( ‘MM/DD/YY HH:MM:SS‘) into a datetime object
try:
    dateTimeObj = datetime.strptime(dateTimeString, '%d-%m-%Y %H:%M:%S')
    print(dateTimeObj)
except ValueError as e:
    print(e)

Output :
time data '5/15/2021 11:12:13' does not match format '%d-%m-%Y %H:%M:%S'

Python: Convert string to datetime – ( string format yyyy-mm-dd hh-mm-ss) :

from datetime import datetime
dateTimeString = '2021-5-15 11-12-13'
#Converting the String ( ‘yyyy-mm-dd hh-mm-ss‘) into a datetime object
dateTimeObj = datetime.strptime(dateTimeString, '%Y-%m-%d %H-%M-%S')
print(dateTimeObj)

Output :
2021-05-15 11:12:13

Python: Convert string to datetime – ( string format MMM DD YYYY HH:MM:SS) :

from datetime import datetime
dateTimeString = 'May 15 2021 11:12:13'
#Converting the String ( ‘MMM DD YYYY HH:MM:SS‘) into a datetime object
dateTimeObj = datetime.strptime(dateTimeString, '%b %d %Y %H:%M:%S')
print(dateTimeObj)

Output :
2021-05-15 11:12:13

Python: Convert string to datetime with milliseconds- ( string format DD/MM/YY HH:MM:SS:FFFFFF) :

In case we have the millisecond info,

from datetime import datetime
dateTimeString = '15/5/21 11:12:13.453'
#Converting the String ( ‘DD/MM/YY HH:MM:SS:FFFFFF‘) into a datetime object
dateTimeObj = datetime.strptime(dateTimeString, '%d/%m/%y %H:%M:%S.%f')
print(dateTimeObj)

Output :
2021-05-15 11:12:13.453000

Python: Convert string to datetime with timezone :

To add the time zone in the format we have to include %z

from datetime import datetime
dateTimeString = '15/5/21 11:12:13+05:30'
#Converting the String wih timezone into a datetime object
dateTimeObj = datetime.strptime(dateTimeString, '%d/%m/%y %H:%M:%S%z')
print(dateTimeObj)

Output :
2021-05-15 11:12:13+05:30

Python: Convert string to date object :

To convert a datetime string into a date object we have to first convert it into a datetime object using strptime( ) and then pass it into the date( ) function to get our date object.

from datetime import datetime
dateTimeString = '2021-5-15'
#Converting the String into a date object
dateTimeObj = datetime.strptime(dateTimeString, '%Y-%m-%d').date()
print(dateTimeObj)

Output :
2021-05-15

Python: Convert string to time object :

To convert a datetime string into a time object we have to first convert it into a datetime object using strptime( ) and then pass it into the time( ) function to get our time object.

from datetime import datetime
dateTimeString = '11:12:13'
#Converting the String into a time object
dateTimeObj = datetime.strptime(dateTimeString, '%H:%M:%S').time()
print(dateTimeObj)

Output :
11:12:13