split_sentences - handling spaces after "."

[Ayushk4]

It might be better if the empty second element in resulting array isn't there.

julia> WordTokenizers.split_sentences("This is a sentence. ")
2-element Array{SubString{String},1}:
 "This is a sentence."
 "" 

[oxinabox]

I would be in favor of this

[RohitPingale]

julia> WordTokenizers.split_sentences("This is a sentence. ")
SubString{String}["This is a sentence.", ""]
How about filtering the "" elements from the array?

[oxinabox]

I think we just change the regex so it never appears in the first place

[RohitPingale]

sentences = replace(sentences, r"([?!.])\s" => Base.SubstitutionString("\\1\n"))
The problem is in this line. The regex is working fine but while SubstitutionString it putting the \n which is needed if we have two or more sentences because we are splitting the sentences followed by this sentences = split(sentences, "\n"). But in above case \n not needed because its just a white space.

[oxinabox]

Link to PR: #37

[RohitPingale]

julia>WordTokenizers.split_sentences(" This is a sentence.Laugh Out Loud. Keep coding. No. Yes! True! ohh!ya! me too. ")
7-element Array{SubString{String},1}:
" This is a sentence.Laugh Out Loud."
"Keep coding."
"No."
"Yes!"
"True!"
"ohh!ya!"
"me too."
I observed that the sentence which has no space after delimiter(Obviously that sentence grammatically incorrect) is not considered as two separate sentences(Like .Laugh Out Loud. and Ohh!ya!). Can this consider as an issue?

[oxinabox]

Yes, done.