Turkish Spell Checker

This tool is a spelling checker for Modern Turkish. It detects spelling errors and corrects them appropriately, through its list of misspellings and matching to the Turkish dictionary.

Simple Web Interface

Link 1 Link 2

Video Lectures

For Developers

You can also see Cython, Php, Java, C++, C, Swift, Js, or C# repository.

Requirements

• Python 3.7 or higher
• Git

Python

To check if you have a compatible version of Python installed, use the following command:

python -V

You can find the latest version of Python here.

Git

Install the latest version of Git.

Pip Install

pip3 install NlpToolkit-SpellChecker

Download Code

In order to work on code, create a fork from GitHub page. Use Git for cloning the code to your local or below line for Ubuntu:

git clone <your-fork-git-link>

A directory called SpellChecker will be created. Or you can use below link for exploring the code:

git clone https://github.com/starlangsoftware/TurkishSpellChecker-Py.git

Open project with Pycharm IDE

Steps for opening the cloned project:

• Start IDE
• Select File | Open from main menu
• Choose DataStructure-PY file
• Select open as project option
• Couple of seconds, project will be downloaded.

Detailed Description

• Creating SpellChecker
• Spell Correction

Creating SpellChecker

SpellChecker finds spelling errors and corrects them in Turkish. There are two types of spell checker available:

• SimpleSpellChecker

  • To instantiate this, a FsmMorphologicalAnalyzer is needed.

      fsm = FsmMorphologicalAnalyzer()
      spellChecker = SimpleSpellChecker(fsm)   
    

• NGramSpellChecker,

  • To create an instance of this, both a FsmMorphologicalAnalyzer and a NGram is required.

  • FsmMorphologicalAnalyzer can be instantiated as follows:

      fsm = FsmMorphologicalAnalyzer()
    

  • NGram can be either trained from scratch or loaded from an existing model.

    • Training from scratch:

        corpus = Corpus("corpus.txt");
        ngram = NGram(corpus.getAllWordsAsArrayList(), 1)
        ngram.calculateNGramProbabilities(LaplaceSmoothing())
      

    There are many smoothing methods available. For other smoothing methods, check here.

    • Loading from an existing model:

        ngram = NGram("ngram.txt")
      

  For further details, please check here.

  • Afterwards, NGramSpellChecker can be created as below:

      spellChecker = NGramSpellChecker(fsm, ngram)
    

Spell Correction

Spell correction can be done as follows:

sentence = Sentence("Dıktor olaç yazdı")
corrected = spellChecker.spellCheck(sentence)
print(corrected)

Output:

Doktor ilaç yazdı

For Contibutors

Setup.py file

1. Do not forget to set package list. All subfolders should be added to the package list.

    packages=['Classification', 'Classification.Model', 'Classification.Model.DecisionTree',
              'Classification.Model.Ensemble', 'Classification.Model.NeuralNetwork',
              'Classification.Model.NonParametric', 'Classification.Model.Parametric',
              'Classification.Filter', 'Classification.DataSet', 'Classification.Instance', 'Classification.Attribute',
              'Classification.Parameter', 'Classification.Experiment',
              'Classification.Performance', 'Classification.InstanceList', 'Classification.DistanceMetric',
              'Classification.StatisticalTest', 'Classification.FeatureSelection'],

2. Package name should be lowercase and only may include _ character.

    name='nlptoolkit_math',

Python files

1. Do not forget to comment each function.

    def __broadcast_shape(self, shape1: Tuple[int, ...], shape2: Tuple[int, ...]) -> Tuple[int, ...]:
        """
        Determines the broadcasted shape of two tensors.

        :param shape1: Tuple representing the first tensor shape.
        :param shape2: Tuple representing the second tensor shape.
        :return: Tuple representing the broadcasted shape.
        """

2. Function names should follow caml case.

    def addItem(self, item: str):

3. Local variables should follow snake case.

	det = 1.0
	copy_of_matrix = copy.deepcopy(self)

4. Class variables should be declared in each file.

class Eigenvector(Vector):
    eigenvalue: float

5. Variable types should be defined for function parameters and class variables.

    def getIndex(self, item: str) -> int:

6. For abstract methods, use ABC package and declare them with @abstractmethod.

    @abstractmethod
    def train(self, train_set: list[Tensor]):
        pass

7. For private methods, use __ as prefix in their names.

    def __infer_shape(self, data: Union[List, List[List], List[List[List]]]) -> Tuple[int, ...]:

8. For private class variables, use __ as prefix in their names.

class Matrix(object):
    __row: int
    __col: int
    __values: list[list[float]]

9. Write __repr__ class methods as toString methods
10. Write getter and setter class methods.

    def getOptimizer(self) -> Optimizer:
        return self.optimizer
    def setValue(self, value: Optional[Tensor]) -> None:
        self._value = value

11. If there are multiple constructors for a class, define them as constructor1, constructor2, ..., then from the original constructor call these methods.

    def constructor1(self):
        self.__values = []
        self.__size = 0

    def constructor2(self, values: list):
        self.__values = values.copy()
        self.__size = len(values)

    def __init__(self,
                 valuesOrSize=None,
                 initial=None):
        if valuesOrSize is None:
            self.constructor1()
        elif isinstance(valuesOrSize, list):
            self.constructor2(valuesOrSize)

12. Extend test classes from unittest and use separate unit test methods.

class TensorTest(unittest.TestCase):

    def test_inferred_shape(self):
        a = Tensor([[1.0, 2.0], [3.0, 4.0]])
        self.assertEqual((2, 2), a.getShape())

    def test_shape(self):
        a = Tensor([1.0, 2.0, 3.0])
        self.assertEqual((3, ), a.getShape())

13. Enumerated types should be used when necessary as enum classes.

class AttributeType(Enum):
    """
    Continuous Attribute
    """
    CONTINUOUS = auto()
    """
    Discrete Attribute
    """
    DISCRETE = auto()