Is there a way to remove punctuation from Persian text?

I want to get rid of punctuations from my text file which is an English-Persian sentence pairs data.

I have tried the following code:

import string
import re
from numpy import array, argmax, random, take
import pandas as pd

# function to read raw text file
def read_text(filename):
    # open the file
    file = open(filename, mode='rt', encoding='utf-8')

    # read all text
    text = file.read()
    file.close()
    return text

# split a text into sentences
def to_lines(text):
  sents = text.strip().split('\n')
  sents = [i.split('\t') for i in sents]
  return sents


data = read_text("pes.txt")
pes_eng = to_lines(data)
pes_eng = array(pes_eng)

# Remove punctuation
pes_eng[:,0] = [s.translate(str.maketrans('', '', string.punctuation)) for s         
in pes_eng[:,0]]
pes_eng[:,1] = [s.replace("؟!.،,?" ,"") for s in pes_eng]

print(pes_eng)

the code above works with English sentences but it is not doing anything with Persian sentences.

Here the output is:

Traceback (most recent call last):
  File ".\persian_to_english.py", line 29, in <module>
    pes_eng[:,1] = [s.replace("؟!.،,?" ,"") for s in pes_eng]
  File ".\persian_to_english.py", line 29, in <listcomp>
    pes_eng[:,1] = [s.replace("؟!.،,?" ,"") for s in pes_eng]
AttributeError: 'numpy.ndarray' object has no attribute 'replace'

But what I want is something like this:

['Who' 'چه کسی']

Solution 1:

You can use list comprehension to create a new list containing what you want:

new_pes_eng = [s.replace("؟!.،,?" ,"") for s in pes_eng]

The line above removes punctuation marks (the ones in first argument passed to replace()) if there is any, from your pes_eng list items.