SPRKPY1047
pyspark.context.SparkContext.setLogLevel has workaround
This issue appears when the tool detects the usage of pyspark.context.SparkContext.setLogLevel which has a workaround.
Input code:
sparkSession.sparkContext.setLogLevel("WARN")
Output code:
#EWI: SPRKPY1047 => pyspark.context.SparkContext.setLogLevel has a workaround, see documentation for more info
sparkSession.sparkContext.setLogLevel("WARN")
setLogLevel(self, logLevel: str) -> None
Action: Replace the "setLogLevel" function usage with "logging.basicConfig" that provides a set of convenience functions for simple logging usage. In order to use it, we need to import two modules, "logging" and "sys", and the level constant should be replaced using the "Level equivalent table":
import logging
import sys
logging.basicConfig(stream=sys.stdout, level=logging.WARNING)
Level source parameter | Level target parameter |
|---|---|
"ALL" | This has no equivalent |
"DEBUG" | logging.DEBUG |
"ERROR" | logging.ERROR |
"FATAL" | logging.CRITICAL |
"INFO" | logging.INFO |
"OFF" | logging.NOTSET |
"TRACE" | This has no equivalent |
"WARN" | logging.WARNING |
- If you need more support, you can email us at [email protected] or post a message to our forums. If you have a contract for support with Mobilize.Net, reach out to your sales engineer and they can direct your support needs.
Last modified 2mo ago