Release Notes
Updates on each released version of Mobilize.Net SnowConvert for Spark
Scala 0.1.468
SparkSnowConvert Core 1.0.23
Added:
- Symbol resolution for function calls without parentheses
- Scopes opening/closing exceptions handling(at Replacers)
- EWI generation for not supported imports (complex cases)
- EWI generation for not defined imports
- SparkSession transformation improvements
- DataFrame reader/writer transformation improvements
- "Spark Usages by Support Category", "Scala Import Call Summary" sections added to Detailed report
- RDD mappings update
Fixed:
- Stack Overflow, output files were not generated
- Expression without parentheses on Spark Session replacer transformation
Scala 0.1.458
SparkSnowConvert Core 0.1.530
Added:
- Updated helper/extension .jar to latest version
- Updated assessment .docx report template
- Import usages inventory generation
- Generating EWIs for not supported imports (simple case)
Fixed:
- Indeterminism issue on SymblTable
- Error when sorting spark usages inventory files
- SclSingleExprPath must not contain null members
- The collection was modified; the enumeration operation may not execute
- Parsing does not finish when there are multiple closing multi-line in a row
- Issue with expression
- Error FileNotGenerated
Scala 0.1.442
SparkSnowConvert Core 0.1.499
Fixed:
- The setting button is not refreshing when the license is changed.
Scala 0.1.442
SparkSnowConvert Core 0.1.498
Added:
- Symbol table built ins loading improvements
- Adding robustness to symbol table loaders
Fixed:
- Error in the total of Scala files in the AssessmentReport
- Symbol resolving for generic functions using an asterisk
- Comments inside comments and id prefix and interpolation parsing error
- The comma after identifier parsing error
- Parsing error of the expression when the first statement is taking the pattern of the second statement
- "and", "::","++" and "or" operators parsing errors
Scala 0.1.430
SparkSnowConvert Core 0.1.491.0
Added
- Symbol loading/resolving - Add support for generic methods with asterisk params.
- Symbol loading/resolving - Add type inference for type defs.
- Symbol loading/resolving general improvements
Fixed
Issue related to the import usages not being stored if there are no Spark references.
Scala 0.1.427
SparkSnowConvert Core 0.1.486.0
Added
- Cloud telemetry and sending email mechanism now available in Conversion Mode
- Update contact information in the email template
Scala 0.1.426
SparkSnowConvert Core 0.1.476.0
Added
- 'SnowConvert Version' and 'Snowpark version' columns to SparkUsagesInventory
- Improvements to speed analysis
Scala 0.1.422
SparkSnowConvert Core 0.1.454.0
Added
- Automated and Status columns added to SparkReferenceInventory.csv
- Summary and detailed html report uploading to Snowflake
- Mappings update
Fixed:
- Summary and detailed report wordings fixes
- Email template wording fixes.
Scala 0.1.421 Spark
SnowConvert Core 0.1.414
Added - Email template update
Adding "Version information" section to Summary report
Adding "Resources" section to Detailed report
Final screen UI changes
Fixed:
Report missing spark functions on sparkUsagesInventory.pam
Detailed report logos update
Percentage values precision on summary and detailed assessment reports
Scala 0.1.421
SparkSnowConvert Core 0.1.396
Added Spark read and write transformations improvement
Adding session id column to spark usages inventory
Scala 0.1.411
SparkSnowConvert Core 0.1.279
Added
- Spark read and write transformations
- Spark trim, rtrim and ltrim function transformations
- String interpolation parsing
- Increasing sql extraction match patterns
Scala 0.1.402
SparkSnowConvert Core 0.1.274
Added
- File operations robustness
- Output folders reorganization
- SparkSession builder transformation
- Adding "Scala files with embedded sql" count in assessment reports
Fixed
- Cyclic dependencies issue on Symbol Table
- Empty case clause parsing
- Multiple statements on lambda block parsing
- Case clause pattern parsing
Scala 0.1.391
SparkSnowConvert Core 0.1.229
Added
- Parsing robustness
- .sbt configuration files processing
- Issues breakdown section added in assessment html report
- Look and feel improvements in assessment html report
- Using RapidScanner inventories to calculate the spark usages assessment
- macOS CLI & UI support
- Improvements in import statements mappings
Scala 0.1.380
Added
- Scala Parser
- Double exclamation mark support
- Conversion tool
- Sql extraction
- object_struct function transformation
- avg function transformation
- Snowpark extensions .jar update
- Lines of code report
- Import mappings
- Docx and html assessment reports
- RapidScan integration
- Linux OS support
Fixed
- Binary expressions special cases parsing
Scala 0.1.358
Added
- Scala Parser
- Support underscore followed by newline when parsing expressions
- Improve parsing errors handling
- Symbols
- Improve support of Unresolved Symbols
- Improve creation of Generic Symbols to reuse existing ones
- Support Loading and Resolution of Lambda Expressions
- Mappings:
- Support custom mappings for functions and types via .map files
- Added custom map directory parameter
Fixed
- Fill missing columns at notification .pam file.
- Generate metrics data files (.pam) to specified reports folder
Added
- Updated logos and text in UI and Documentation
- Symbols
- Support Generic Identifiers on Type Parameters for Generic Symbol
- Exclusion of not required dependencies
- ScalaParser:
- Backticks idents
- ArgAssign expressions
Fixed
- ScalaParser:
- ExprLambda with ColonType next to ident
- Try expression when try is not referring a keyword
- Empty lambda expr with args
- Underscore ("_") in TypeArgs
- Files with all commented out source
- New lines at SimpleExpr, SingleExpr, TailExpr nodes
- ConversionTool:
- Fix Crash of conversion due to javap parsing errors (related with jar dependencies)
Features
- Command line interface.
- Scala code assessment feature.
- Consume multiple files or single files with multiple objects.
- Conversion of basic Scala programs as defined by functions and syntax to be mutually agreed during the first 3 development sprints.
- Comments in Scala code are re-inserted inline.
- Insert comments in-line with any errors/warning/reviews.
- Basic reporting including
- Number of spark elements processed
- Summary of elements transformed, files and locations of
- Summary of errors/warnings/reviews encountered.
- Summary of unsupported Spark APIs
- Demonstrated inclusion of the following defined scenarios:
- API mappings
- Recreate project as SnowPark projects
- Setup Proper project structure
- Update to SnowPark supported Scala version
- Helper Creation to reduce impedance mismatch
- Define some pattern rewrite
- Document guidelines for non-automatable concepts (e.g.: file usage patterns, data source configuration, or spark libraries without a direct equivalent, like Kafka stream reading)
- Greater than 90% successful conversion rate for initial two customer code bases (basis code for the above scenarios) to be provided to Mobilize by Snowflake on the Effective Date.
- Measured based upon number of compilable objects in Snowflake
- Objects with unsupported/untranslatable functions not counted
- Conversion rate for code will be based upon a complete code base containing all dependent objects.
- Snowflake will provide access to all available private preview features for Mobilize development benefit
Last modified 13d ago