Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
108 commits
Select commit Hold shift + click to select a range
23ea9e5
remove imports after move
Jan 23, 2025
833c7a8
Merge remote-tracking branch 'origin/develop' into tom-cg-7930-remove…
Jan 30, 2025
2542923
merge
Jan 30, 2025
abb5bc6
return rmd code
Jan 30, 2025
c65b534
improve python import resolution etc
Jan 30, 2025
7670afa
py test fix
Jan 31, 2025
004cf06
typescript UTs and code changes for new edge cases
Jan 31, 2025
5556510
UT fixes
Feb 4, 2025
daf1b75
Automated pre-commit update
tkfoss Feb 4, 2025
5da1428
rm prints
Feb 4, 2025
57577ff
Merge branch 'develop' into tom-cg-7930-remove-now-unused-imports-aft…
EdwardJXLi Feb 5, 2025
32769df
Fix tests for Python `test_function_move_to_file`
EdwardJXLi Feb 5, 2025
6b7e688
Fix tests for Typescript `test_function_move_to_file`
EdwardJXLi Feb 5, 2025
f744bf2
Remove apidoc from remove_unused_imports and remove_unused_exports
EdwardJXLi Feb 5, 2025
7123cf9
Remove py_apidoc from is_from_import
EdwardJXLi Feb 5, 2025
6a31479
simplify py code
Feb 6, 2025
4aa7676
Feature flag generics support (#304)
bagel897 Feb 5, 2025
682c428
Specify language on 'codegen init' in CLI (#289)
vishalshenoy Feb 5, 2025
f6290cf
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 5, 2025
3c33141
Fix: duplicate edge creation (#305)
bagel897 Feb 5, 2025
62e1176
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 5, 2025
5ef9f27
chore(ci): CG-10672 add back 3.13 mac build (#302)
christinewangcw Feb 5, 2025
624295c
ci: don't report coverage data as json (#307)
bagel897 Feb 5, 2025
b22c368
fix: pre-commit on develop branch (#309)
christinewangcw Feb 5, 2025
fbeb41b
fix: pre-commit missing `--source` (#312)
christinewangcw Feb 5, 2025
956d3ea
docs: Add docs for incremental recomputation (#311)
bagel897 Feb 5, 2025
01f2fa1
chore(deps): update dependency aws-cli to v5.1.4 (#310)
renovate[bot] Feb 5, 2025
b209125
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 5, 2025
6af57d8
chore(deps): update dependency aws-cli to v5.2.0 (#314)
renovate[bot] Feb 5, 2025
c7b4401
docs: add Python 3.13 recommendation to README (#303)
devin-ai-integration[bot] Feb 5, 2025
310891f
chore(ci): [CG-10689] add slack alert in release (#316)
christinewangcw Feb 5, 2025
658b010
Ignore folder (#317)
eacodegen Feb 5, 2025
50d40d9
chore(ci): clean-up circle ci workflows (#319)
christinewangcw Feb 5, 2025
4e0a66e
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 6, 2025
13c9ec4
Set default value (#322)
bagel897 Feb 6, 2025
737e555
Mypyc/cython changes (#318)
bagel897 Feb 6, 2025
b07e6f2
fix bug (#323)
bagel897 Feb 6, 2025
af179b5
fix: empty collection remove (#324)
bagel897 Feb 6, 2025
75dd7dc
fix(ci): invalid gh template (#320)
christinewangcw Feb 6, 2025
589d089
chore(ci): add issue comment for arm + remove install deps (#325)
christinewangcw Feb 6, 2025
2820313
Fix JSX prop parsing (#326)
bagel897 Feb 6, 2025
676ce46
feat(ci) CG-10496: semantic release (#328)
christinewangcw Feb 6, 2025
bf7b7d0
chore: separate workflow for semantic (#329)
christinewangcw Feb 6, 2025
6de94d3
chore(ci): delete circle CI validate hook (#330)
christinewangcw Feb 6, 2025
77ab8b9
chore: add workflow dispatch to auto release (#331)
christinewangcw Feb 6, 2025
cce834c
chore(deps): update pre-commit hook astral-sh/uv-pre-commit to v0.5.2…
renovate[bot] Feb 6, 2025
dc04179
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 6, 2025
9f1b9ae
chore(ci): set build skip in pyproject (#337)
christinewangcw Feb 6, 2025
56eceff
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 6, 2025
48c3ffb
git
Feb 6, 2025
641b0aa
test remove unused imports
frainfreeze Feb 7, 2025
fe36690
Automated pre-commit update
tkfoss Feb 7, 2025
d71fca8
UTs & export refactor
frainfreeze Feb 7, 2025
a225999
Automated pre-commit update
tkfoss Feb 7, 2025
cee50b2
revert github workflow file, py code, ut changes
frainfreeze Feb 7, 2025
8ca64ff
Automated pre-commit update
tkfoss Feb 7, 2025
d2f3414
update
Feb 10, 2025
37664b9
chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.9…
renovate[bot] Feb 6, 2025
bfbd7b0
Remove og image default (#345)
joelaguero Feb 6, 2025
ef987af
feat(docs): Changelog generation (#341)
jemeza-codegen Feb 6, 2025
34cdf58
Foundations for PR BOT static analisis (#343)
kopekC Feb 6, 2025
e429498
chore(deps): update dependency httpx to <0.28.2,>=0.28.1 (#346)
renovate[bot] Feb 6, 2025
440a57f
Tawsif fix asyncify promise return type (#348)
tawsifkamal Feb 6, 2025
636034a
CG-10694: Remove lowside + enterprise from codegen.git (#349)
caroljung-cg Feb 6, 2025
609c0b6
fix: Disable uv cache (#351)
caroljung-cg Feb 7, 2025
fdfe835
Fix GithubClient constructor (#352)
caroljung-cg Feb 7, 2025
36cf065
chore: Remove access token from local repo operator (#354)
caroljung-cg Feb 7, 2025
14595fb
Changes to default urls (#357)
kopekC Feb 7, 2025
d4fe603
chore: subdir logging (#356)
christinewangcw Feb 7, 2025
516d9c1
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 7, 2025
2bd1e4a
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 7, 2025
2e0449d
feat: [CG-10632] mcp server (#358)
rushilpatel0 Feb 7, 2025
ba4b842
docs: remove overview iframe (#363)
jayhack Feb 7, 2025
ac2c348
fix init.py bug (#364)
bagel897 Feb 7, 2025
547eef0
Experiment: Override `__class__` for Builtin types to shadow `isinsta…
EdwardJXLi Feb 7, 2025
a03ba1c
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 7, 2025
ee5a854
chore: disable failing parse tests for now (#366)
christinewangcw Feb 7, 2025
2995692
docs: switches main button to be github (#373)
jayhack Feb 8, 2025
db70491
docs: removes code link backticks (#369)
jayhack Feb 8, 2025
4abaae0
chore(deps): update pre-commit hook renovatebot/pre-commit-hooks to v…
renovate[bot] Feb 9, 2025
f55f27c
[WIP] Langchain demo (#374)
jayhack Feb 9, 2025
106111e
feat: adds VectorIndex extension (#378)
jayhack Feb 9, 2025
6224ca6
chore(deps): lock file maintenance (#379)
renovate[bot] Feb 10, 2025
df1e801
[WIP] Modal Demo (#381)
jayhack Feb 10, 2025
38aa188
docs: adds code agent tutorial (#382)
jayhack Feb 10, 2025
fe2a026
chore(deps): lock file maintenance (#384)
renovate[bot] Feb 10, 2025
eb38163
codegen-examples is dead, long live codegen-examples (#375)
kopekC Feb 10, 2025
73b93a6
chore(deps): update dependency jupyterlab to v4.3.5 (#385)
renovate[bot] Feb 10, 2025
1fec4b3
fix(deps): update dependency codegen to v0.5.30 (#387)
renovate[bot] Feb 10, 2025
dd54bf9
docs: langchain + modal examples (#386)
jayhack Feb 10, 2025
e36d0b6
[wip] Modal RAG example (#388)
jayhack Feb 10, 2025
d863ad3
chore(deps): update pre-commit hook astral-sh/ruff-pre-commit to v0.9…
renovate[bot] Feb 10, 2025
4e5f74a
fix: remove pre-push hook on auto release (#390)
christinewangcw Feb 10, 2025
2763872
CG-10301: renaming file path bug (#392)
Feb 10, 2025
4f17fba
build: Support x86_64 mac (#393)
eacodegen Feb 10, 2025
85a4331
Fix OSS Parse Tests (#372)
EdwardJXLi Feb 10, 2025
029fcfa
Update plotly requirement from <6.0.0,>=5.24.0 to >=5.24.0,<7.0.0 (#246)
dependabot[bot] Feb 10, 2025
4408ae8
fix: wait for checks semantic release (#395)
christinewangcw Feb 10, 2025
cf24fbd
CG-10731: Add ChainedAttribute.attribute_chain (#383)
tawsifkamal Feb 10, 2025
1332f95
fix CG-9440 clean repo - clears from the default branch (#398)
christinewangcw Feb 10, 2025
3524117
chore(testing): set base dir in op creation (#399)
christinewangcw Feb 10, 2025
ef9881e
CG-10470: Add `config` CLI commands (#391)
caroljung-cg Feb 10, 2025
428b289
Remove LFS from `codegen-sdk` (+ disable `disallowed-words` check) (#…
EdwardJXLi Feb 10, 2025
37eb5bb
docs: remove Apple siliicon (#368)
jayhack Feb 10, 2025
e03e067
ops: disable auto-release (#400)
christinewangcw Feb 10, 2025
2fb7d06
Automated pre-commit update
tkfoss Feb 10, 2025
2642cca
Merge branch 'develop' into tom-cg-7930-remove-now-unused-imports-aft…
Feb 10, 2025
2ef105c
Merge branch 'develop' into tom-cg-7930-remove-now-unused-imports-aft…
Feb 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 43 additions & 2 deletions src/codegen/sdk/core/symbol.py
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,7 @@ def move_to_file(
file: SourceFile,
include_dependencies: bool = True,
strategy: Literal["add_back_edge", "update_all_imports", "duplicate_dependencies"] = "update_all_imports",
remove_unused_imports: bool = True,
) -> None:
"""Moves the given symbol to a new file and updates its imports and references.

Expand All @@ -282,6 +283,7 @@ def move_to_file(
strategy (str): The strategy to use for updating imports. Can be either 'add_back_edge' or 'update_all_imports'. Defaults to 'update_all_imports'.
- 'add_back_edge': Moves the symbol and adds an import in the original file
- 'update_all_imports': Updates all imports and usages of the symbol to reference the new file
remove_unused_imports (bool): If True, removes any imports in the original file that become unused after moving the symbol. Defaults to True.

Returns:
None
Expand All @@ -290,7 +292,7 @@ def move_to_file(
AssertionError: If an invalid strategy is provided.
"""
encountered_symbols = {self}
self._move_to_file(file, encountered_symbols, include_dependencies, strategy)
self._move_to_file(file, encountered_symbols, include_dependencies, strategy, remove_unused_imports)

@noapidoc
def _move_to_file(
Expand All @@ -299,16 +301,21 @@ def _move_to_file(
encountered_symbols: set[Symbol | Import],
include_dependencies: bool = True,
strategy: Literal["add_back_edge", "update_all_imports", "duplicate_dependencies"] = "update_all_imports",
remove_unused_imports: bool = True,
) -> tuple[NodeId, NodeId]:
"""Helper recursive function for `move_to_file`"""
from codegen.sdk.core.import_resolution import Import

# Track original file and imports used by this symbol before moving
symbol_imports = set()

# =====[ Arg checking ]=====
if file == self.file:
return file.file_node_id, self.node_id
if imp := file.get_import(self.name):
encountered_symbols.add(imp)
imp.remove()
if remove_unused_imports:
imp.remove()

if include_dependencies:
# =====[ Move over dependencies recursively ]=====
Expand Down Expand Up @@ -391,6 +398,40 @@ def _move_to_file(
# Delete the original symbol
self.remove()

# After moving a symbol (function or class) out of a file, if there are imports that are now unused because that was the only thing using them, remove those as well
if remove_unused_imports:
# Get all imports that were used by the moved symbol
for dep in self.dependencies:
if isinstance(dep, Import):
symbol_imports.add(dep)

# Check each import that was used by the moved symbol
for import_symbol in symbol_imports:
try:
# Try to access any property - if the import was removed this will fail
_ = import_symbol.file
except (AttributeError, ReferenceError):
# Skip if import was already removed
continue

# Check if import is still used by any remaining symbols
still_used = False
for usage in import_symbol.usages:
# Skip usages from the moved symbol
if usage.usage_symbol == self:
continue

# Skip usages from symbols we moved
if usage.usage_symbol in encountered_symbols:
continue

still_used = True
break

# Remove import if it's no longer used
if not still_used:
import_symbol.remove()

@property
@reader
@noapidoc
Expand Down
34 changes: 31 additions & 3 deletions src/codegen/sdk/python/file.py
Original file line number Diff line number Diff line change
Expand Up @@ -174,10 +174,38 @@ def add_import_from_import_string(self, import_string: str) -> None:
else:
self.insert_before(import_string, priority=1)

@noapidoc
def remove_unused_imports(self) -> None:
"""Removes unused imports from the file.

Handles different Python import styles:
- Single imports (import x)
- From imports (from y import z)
- Multi-imports (from y import (a, b as c))
- Wildcard imports (from x import *)
- Type imports (from typing import List)
- Future imports (from __future__ import annotations)

Preserves:
- Comments and whitespace where possible
- Future imports even if unused
- Type hints and annotations
"""
# Process each import statement
for import_stmt in self.imports:
# Always preserve __future__ and star imports since we can't track their usage
if import_stmt.is_future_import or import_stmt.is_wildcard_import():
continue

import_stmt.remove_if_unused()

self.G.commit_transactions()

def remove_unused_exports(self) -> None:
Comment thread
tkfoss marked this conversation as resolved.
"""Removes unused exports from the file. NO-OP for python"""
pass
"""Removes unused exports from the file.
In Python this is equivalent to removing unused imports since Python doesn't have
explicit export statements. Calls remove_unused_imports() internally.
"""
self.remove_unused_imports()

@cached_property
@noapidoc
Expand Down
49 changes: 49 additions & 0 deletions src/codegen/sdk/python/import_resolution.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,55 @@ def get_import_string(
else:
return f"from {import_module} import {self.name}"

@property
def module_name(self) -> str:
"""Gets the module name for this import.

For 'import x' returns 'x'
For 'from x import y' returns 'x'
For 'from .x import y' returns '.x'

Returns:
str: The module name for this import.
"""
if self.ts_node.type == "import_from_statement":
module_node = self.ts_node.child_by_field_name("module_name")
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rely on self.module?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean use self.module instead of the .module_name? it seems to miss some imports then. or should module_name itself be changed?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's missing imports, that may be a bug in the SDK. Generally we want to use our nodes rather than tree-sitters post-parse

return module_node.text.decode("utf-8") if module_node else ""
return self.ts_node.child_by_field_name("name").text.decode("utf-8")

def is_from_import(self) -> bool:
"""Determines if this is a from-style import statement.

Checks if the import uses 'from' syntax (e.g., 'from module import symbol')
rather than direct import syntax (e.g., 'import module').

Returns True for imports like:
- from x import y
- from .x import y
- from x import (a, b, c)
- from x import *

Returns False for:
- import x
- import x as y

Returns:
bool: True if this is a from-style import, False otherwise.
"""
return self.import_type in [ImportType.NAMED_EXPORT, ImportType.WILDCARD]

@property
def is_future_import(self) -> bool:
"""Determines if this is a __future__ import.

Returns True for imports like:
- from __future__ import annotations

Returns:
bool: True if this is a __future__ import, False otherwise
"""
return self.ts_node.type == "future_import_statement"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we use import_type?



class PyExternalImportResolver(ExternalImportResolver):
def __init__(self, from_alias: str, to_context: CodebaseGraph) -> None:
Expand Down
156 changes: 98 additions & 58 deletions src/codegen/sdk/typescript/file.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
from codegen.sdk.core.autocommit import commiter, mover, reader, writer
from codegen.sdk.core.file import SourceFile
from codegen.sdk.core.interfaces.exportable import Exportable
from codegen.sdk.enums import ImportType, NodeType, ProgrammingLanguage, SymbolType
from codegen.sdk.enums import ImportType, ProgrammingLanguage, SymbolType
from codegen.sdk.extensions.sort import sort_editables
from codegen.sdk.extensions.utils import cached_property
from codegen.sdk.typescript.assignment import TSAssignment
Expand Down Expand Up @@ -238,63 +238,6 @@ def _parse_imports(self) -> None:
if function.type == "import" or (function.type == "identifier" and function.text.decode("utf-8") == "require"):
TSImportStatement(import_node, self.node_id, self.G, self.code_block, 0)

@writer
def remove_unused_exports(self) -> None:
"""Removes unused exports from the file.

Analyzes all exports in the file and removes any that are not used. An export is considered unused if it has no direct
symbol usages and no re-exports that are used elsewhere in the codebase.

When removing unused exports, the method also cleans up any related unused imports. For default exports, it removes
the 'export default' keyword, and for named exports, it removes the 'export' keyword or the entire export statement.

Args:
None

Returns:
None
"""
for export in self.exports:
symbol_export_unused = True
symbols_to_remove = []

exported_symbol = export.resolved_symbol
for export_usage in export.symbol_usages:
if export_usage.node_type == NodeType.IMPORT or (export_usage.node_type == NodeType.EXPORT and export_usage.resolved_symbol != exported_symbol):
# If the import has no usages then we can add the import to the list of symbols to remove
reexport_usages = export_usage.symbol_usages
if len(reexport_usages) == 0:
symbols_to_remove.append(export_usage)
break

# If any of the import's usages are valid symbol usages, export is used.
if any(usage.node_type == NodeType.SYMBOL for usage in reexport_usages):
symbol_export_unused = False
break

symbols_to_remove.append(export_usage)

elif export_usage.node_type == NodeType.SYMBOL:
symbol_export_unused = False
break

# export is not used, remove it
if symbol_export_unused:
# remove the unused imports
for imp in symbols_to_remove:
imp.remove()

if exported_symbol == exported_symbol.export.declared_symbol:
# change this to be more robust
if exported_symbol.source.startswith("export default "):
exported_symbol.replace("export default ", "")
else:
exported_symbol.replace("export ", "")
else:
exported_symbol.export.remove()
if exported_symbol.export != export:
export.remove()

@noapidoc
def _get_export_data(self, relative_path: str, export_type: str = "EXPORT") -> tuple[tuple[str, str], dict[str, callable]]:
quoted_paths = (f"'{relative_path}'", f'"{relative_path}"')
Expand Down Expand Up @@ -394,11 +337,27 @@ def get_import_string(self, alias: str | None = None, module: str | None = None,
def valid_import_names(self) -> dict[str, Symbol | TSImport]:
"""Returns a dict mapping name => Symbol (or import) in this file that can be imported from another file"""
valid_export_names = {}

# Handle default exports
if len(self.default_exports) == 1:
valid_export_names["default"] = self.default_exports[0]

# Handle named exports and their aliases
for export in self.exports:
for name, dest in export.names:
# Track both original name and alias if present
valid_export_names[name] = dest
if hasattr(dest, "alias") and dest.alias:
valid_export_names[dest.alias] = dest

# # Handle imports and their aliases
# for import_stmt in self.imports:
# for name, symbol in import_stmt.imported_symbols.items():
# valid_export_names[name] = symbol
# # Also track the alias if present
# if hasattr(symbol, "alias") and symbol.alias:
# valid_export_names[symbol.alias] = symbol

return valid_export_names

####################################################################################################################
Expand Down Expand Up @@ -445,3 +404,84 @@ def get_namespace(self, name: str) -> TSNamespace | None:
TSNamespace | None: The namespace with the specified name if found, None otherwise.
"""
return next((x for x in self.symbols if isinstance(x, TSNamespace) and x.name == name), None)

@writer
def remove_unused_imports(self) -> None:
"""Removes unused imports from the file.

Handles different TypeScript import styles:
- Single imports (import x from 'y')
- Named imports (import { x } from 'y')
- Multi-imports (import { a, b as c } from 'y')
- Type imports (import type { X } from 'y')
- Side effect imports (import 'y')
- Wildcard imports (import * as x from 'y')

Preserves:
- Comments and whitespace where possible
- Side effect imports (e.g., CSS imports)
- Type imports used in type annotations
"""
# Process each import statement
for import_stmt in self.imports:
# Always preserve side effect imports since we can't track their usage
if import_stmt.import_type == ImportType.SIDE_EFFECT:
continue

# Check if all imports in this statement are unused
import_stmt.remove_if_unused()

self.G.commit_transactions()

def _is_export_used(self, export: TSExport) -> bool:
# Get all symbol usages
usages = export.symbol_usages()

# If there are any usages, the export is used
if usages:
return True

# Check if this is a re-export that's used elsewhere
if export.is_reexport():
# Get the original symbol
original = export.resolved_symbol
if original:
# Check usages of the original symbol
return bool(original.symbol_usages())

return False

@writer
def remove_unused_exports(self) -> None:
"""Removes unused exports from the file.

Handles different TypeScript export styles:
- Default exports (export default x)
- Named exports (export function x, export const x)
- Re-exports (export { x } from 'y')
- Type exports (export type X, export interface X)

Preserves:
- Type exports (these may be used in type positions)
- Default exports (these are often used dynamically)
- Exports used by other files through imports
- Exports used within the same file
"""
exports_to_remove = []

for export in self.exports:
# Skip type exports
if export.is_type_export() or export.is_default_export():
continue

# Check if export is used
if self._is_export_used(export):
continue

exports_to_remove.append(export)

# Remove unused exports
for export in exports_to_remove:
export.remove()

self.G.commit_transactions()
Loading