Errors with sklearn and matrix generation

[jasoncrucify]

Hi, thanks for sharing the package. it sounds amazing.

However, I encountered some problems while using the algorithm from website but also from python environment.

First, I tried both my own dataset and the demo H5ad files, it gives the same type of errors below, do you know why? I also pasted the error log below. it looks like there is a problem of dependency of skilearn, could you provide me which version of skilearn you used in STCAT.

Also, I would like to now, which codes did you use to generate raw matrix in txt and H5ad, I have also tried my own H5ad and txt files from my own project which did not work using your package, but it was ok using other packages like Celltypist or Azimuth, do you have any idea and could you share me this important parameters?

Great thanks !

🧮 T cell number : 2977
🧮 None T cell number : 0
💬 The input file seems not a raw count matrix.
✔ All genes and log1p normalized expression to 10000 counts per cell.

AttributeError Traceback (most recent call last)
File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/models.py:58, in Model.load(model)
57 try:
---> 58 pkl_obj = pickle.load(fh)
59 return Model(pkl_obj['Model'], pkl_obj['Scaler_'], pkl_obj['description'])

AttributeError: Can't get attribute 'Log' on <module 'sklearn.linear_model._sgd_fast' from '/Users/zhicheng/anaconda3/envs/STCAT/lib/python3.9/site-packages/sklearn/linear_model/_sgd_fast.cpython-39-darwin.so'>

During handling of the above exception, another exception occurred:

Exception Traceback (most recent call last)
Cell In[5], line 1
----> 1 results = STCAT.STCAT(adata)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/STCAT.py:59, in STCAT(adata)
57 np.random.seed(0)
58 model = os.path.join(data_path, 'T_cell.pkl')
---> 59 predictions_voting = annotate.annotate(adata_T, model = model, majority_voting = True, min_prop =0.3)
60 df = predictions_voting.predicted_labels.rename(columns={'predicted_labels': 'predicted_labels_'+ str(0), 'over_clustering': 'over_clustering_' + str(0), 'majority_voting': 'majority_voting_'+ str(0)})
61 df = df.reindex(adata_T.obs_names)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/annotate.py:77, in annotate(filename, model, transpose_input, gene_file, cell_file, mode, p_thres, majority_voting, over_clustering, min_prop)
19 """
20 Run the prediction and (optional) majority voting to annotate the input dataset.
21
(...)
74 4) :attr:~celltypist.classifier.AnnotationResult.adata, AnnData representation of the input data.
75 """
76 #load model
---> 77 lr_classifier = model if isinstance(model, Model) else Model.load(model)
78 #construct Classifier class
79 clf = classifier.Classifier(filename = filename, model = lr_classifier, transpose = transpose_input, gene_file = gene_file, cell_file = cell_file)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/models.py:61, in Model.load(model)
59 return Model(pkl_obj['Model'], pkl_obj['Scaler_'], pkl_obj['description'])
60 except Exception as exception:
---> 61 raise Exception(
62 f"🛑 Invalid model: {model}. {exception}")

Exception: 🛑 Invalid model: /Users/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/data/T_cell.pkl. Can't get attribute 'Log' on <module 'sklearn.linear_model._sgd_fast' from '/Users/anaconda3/envs/STCAT/lib/python3.9/site-packages/sklearn/linear_model/_sgd_fast.cpython-39-darwin.so'>

[jasoncrucify]

this is my second error log: I reinstalled again the environment, however, the problem is altered and different: 🧮 T cell number : 2977
🧮 None T cell number : 0
💬 The input file seems not a raw count matrix.
✔ All genes and log1p normalized expression to 10000 counts per cell.
🔬 Input data has 2977 cells and 31584 genes
🔗 Matching reference genes in the model
🧬 3122 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
👀 Can not detect a neighborhood graph, will construct one before the over-clustering
⛓️ Over-clustering input data with resolution set to 1
🗳️ Majority voting the predictions

AttributeError Traceback (most recent call last)
Cell In[4], line 1
----> 1 results = STCAT.STCAT(adata)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/STCAT.py:59, in STCAT(adata)
57 np.random.seed(0)
58 model = os.path.join(data_path, 'T_cell.pkl')
---> 59 predictions_voting = annotate.annotate(adata_T, model = model, majority_voting = True, min_prop =0.3)
60 df = predictions_voting.predicted_labels.rename(columns={'predicted_labels': 'predicted_labels_'+ str(0), 'over_clustering': 'over_clustering_' + str(0), 'majority_voting': 'majority_voting_'+ str(0)})
61 df = df.reindex(adata_T.obs_names)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/annotate.py:106, in annotate(filename, model, transpose_input, gene_file, cell_file, mode, p_thres, majority_voting, over_clustering, min_prop)
103 raise ValueError(
104 f"🛑 Length of over_clustering ({len(over_clustering)}) does not match the number of input cells ({clf.adata.n_obs})")
105 #majority voting
--> 106 return classifier.Classifier.majority_vote(predictions, over_clustering, min_prop = min_prop)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/classifier.py:477, in Classifier.majority_vote(predictions, over_clustering, min_prop)
475 majority = votes.idxmax(axis=0)
476 freqs = (votes / votes.sum(axis=0).values).max(axis=0)
--> 477 majority = majority.cat.add_categories('Heterogeneous')
478 majority[freqs < min_prop] = 'Heterogeneous'
479 majority = majority[over_clustering].reset_index()

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/generic.py:5989, in NDFrame.getattr(self, name)
5982 if (
5983 name not in self._internal_names_set
5984 and name not in self._metadata
5985 and name not in self._accessors
5986 and self._info_axis._can_hold_identifiers_and_holds_name(name)
5987 ):
5988 return self[name]
-> 5989 return object.getattribute(self, name)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/accessor.py:224, in CachedAccessor.get(self, obj, cls)
221 if obj is None:
222 # we're accessing the attribute of the class, i.e., Dataset.geo
223 return self._accessor
--> 224 accessor_obj = self._accessor(obj)
225 # Replace the property with the accessor object. Inspired by:
226 # https://www.pydanny.com/cached-property.html
227 # We need to use object.setattr because we overwrite setattr on
228 # NDFrame
229 object.setattr(obj, self._name, accessor_obj)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/arrays/categorical.py:2445, in CategoricalAccessor.init(self, data)
2444 def init(self, data) -> None:
-> 2445 self._validate(data)
2446 self._parent = data.values
2447 self._index = data.index

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/arrays/categorical.py:2454, in CategoricalAccessor._validate(data)
2451 @staticmethod
2452 def _validate(data):
2453 if not is_categorical_dtype(data.dtype):
-> 2454 raise AttributeError("Can only use .cat accessor with a 'category' dtype")

AttributeError: Can only use .cat accessor with a 'category' dtype

[DawnChou]

this is my second error log: I reinstalled again the environment, however, the problem is altered and different: 🧮 T cell number : 2977

🧮 None T cell number : 0
💬 The input file seems not a raw count matrix.
✔ All genes and log1p normalized expression to 10000 counts per cell.
🔬 Input data has 2977 cells and 31584 genes
🔗 Matching reference genes in the model
🧬 3122 features used for prediction
⚖️ Scaling input data
🖋️ Predicting labels
✅ Prediction done!
👀 Can not detect a neighborhood graph, will construct one before the over-clustering
⛓️ Over-clustering input data with resolution set to 1
🗳️ Majority voting the predictions
AttributeError Traceback (most recent call last) Cell In[4], line 1 ----> 1 results = STCAT.STCAT(adata)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/STCAT.py:59, in STCAT(adata) 57 np.random.seed(0) 58 model = os.path.join(data_path, 'T_cell.pkl') ---> 59 predictions_voting = annotate.annotate(adata_T, model = model, majority_voting = True, min_prop =0.3) 60 df = predictions_voting.predicted_labels.rename(columns={'predicted_labels': 'predicted_labels_'+ str(0), 'over_clustering': 'over_clustering_' + str(0), 'majority_voting': 'majority_voting_'+ str(0)}) 61 df = df.reindex(adata_T.obs_names)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/annotate.py:106, in annotate(filename, model, transpose_input, gene_file, cell_file, mode, p_thres, majority_voting, over_clustering, min_prop) 103 raise ValueError( 104 f"🛑 Length of over_clustering ({len(over_clustering)}) does not match the number of input cells ({clf.adata.n_obs})") 105 #majority voting --> 106 return classifier.Classifier.majority_vote(predictions, over_clustering, min_prop = min_prop)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/STCAT/classifier.py:477, in Classifier.majority_vote(predictions, over_clustering, min_prop) 475 majority = votes.idxmax(axis=0) 476 freqs = (votes / votes.sum(axis=0).values).max(axis=0) --> 477 majority = majority.cat.add_categories('Heterogeneous') 478 majority[freqs < min_prop] = 'Heterogeneous' 479 majority = majority[over_clustering].reset_index()

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/generic.py:5989, in NDFrame.getattr(self, name) 5982 if ( 5983 name not in self._internal_names_set 5984 and name not in self._metadata 5985 and name not in self._accessors 5986 and self._info_axis._can_hold_identifiers_and_holds_name(name) 5987 ): 5988 return self[name] -> 5989 return object.getattribute(self, name)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/accessor.py:224, in CachedAccessor.get(self, obj, cls) 221 if obj is None: 222 # we're accessing the attribute of the class, i.e., Dataset.geo 223 return self._accessor --> 224 accessor_obj = self._accessor(obj) 225 # Replace the property with the accessor object. Inspired by: 226 # https://www.pydanny.com/cached-property.html 227 # We need to use object.setattr because we overwrite setattr on 228 # NDFrame 229 object.setattr(obj, self._name, accessor_obj)

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/arrays/categorical.py:2445, in CategoricalAccessor.init(self, data) 2444 def init(self, data) -> None: -> 2445 self._validate(data) 2446 self._parent = data.values 2447 self._index = data.index

File ~/anaconda3/envs/STCAT/lib/python3.9/site-packages/pandas/core/arrays/categorical.py:2454, in CategoricalAccessor._validate(data) 2451 @staticmethod 2452 def _validate(data): 2453 if not is_categorical_dtype(data.dtype): -> 2454 raise AttributeError("Can only use .cat accessor with a 'category' dtype")

AttributeError: Can only use .cat accessor with a 'category' dtype

Hi, I met the same problem. Have you solved it?