def get_tokens ( line : str ) -> Iterator [ str ] : for token in line . rstrip ( ) . split ( ) : if len ( token ) > 0 : yield token
how to parse a line into tokens in python
Yields tokens from input string .
def get_tokens(line: str) -> Iterator[str]:
    """
    Yields tokens from input string.

    :param line: Input string.
    :return: Iterator over tokens.
    """
    for token in line.rstrip().split():
        if len(token) > 0:
            yield token