r"""Routines for calculation and of solar and sidereal times.

This module can:

- Convert between Python :class:`~datetime.datetime`, UNIX times (as floats),
  and Skyfield time objects.
- Determine the number of leap seconds in an interval (thanks Skyfield!)
- Calculate the Earth Rotation Angle (the successor to Sidereal Time)
- Determine various local time standards (Local Stellar Angle and Day)
- Generate a nice wrapper for Skyfield to ease the loading of required data.

Time Utilities
==============
Time conversion routine which are location independent.

- :py:meth:`unix_to_skyfield_time`
- :py:meth:`datetime_to_unix`
- :py:meth:`unix_to_datetime`
- :py:meth:`datetime_to_timestr`
- :py:meth:`timestr_to_datetime`
- :py:meth:`unix_to_era`
- :py:meth:`era_to_unix`
- :py:meth:`ensure_unix`
- :py:meth:`leap_seconds_between`
- :py:meth:`time_of_day`
- :py:meth:`naive_datetime_to_utc`

Local Time Utilities
====================

Routines which are location specific are grouped into the location aware class :py:class:`Observer`.

This class can be used to calculate Local Stellar Angle (LSA), and the Local
Stellar Day (LSD). LSA is an equivalent to the Local Sidereal Time based around
the Earth Rotation Angle instead of the Greenwich Sidereal Time. This is defined
as:

.. math::
    \mathrm{LSA} = \theta + \lambda

where :math:`\theta` is the Earth Rotation Angle, and :math:`\lambda` is the
longitude. Local Stellar Day counts the stellar days (i.e. the number of cycles
of LSA) that have occured since a given start epoch. This means that the
fractional part is simply the LSA rescaled., with the integer part the number of
stellar days elapsed since that epoch. This requires the specification of the
start epoch, which is determined from a given UNIX time, the code simply picks
the first time :math:`\mathrm{LSA} = 0` after this time.

Note that the quantities LSA and LSD are not really used elsewhere. However, the
concept of a Stellar Day as a length of time is well established (`IERS
constants`_), and the Stellar Angle is an older term for the Earth Rotation
Angle (`NFA Glossary`_).

Skyfield Interface
==================

- :py:class:`SkyfieldWrapper`

This module provides an interface to Skyfield which stores the required datasets
(timescale data and an ephemeris) in a fixed location. The location is
determined by the following (in order):

- As the wrapper is initialised by passing in a ``path=<path>`` option.
- By setting the environment variable ``CAPUT_SKYFIELD_PATH``
- If neither of the above is set, the data is place in ``<path to caput>/caput/data/``

Constants
=========

:const:`SIDEREAL_S`
    Number of SI seconds in a sidereal second [s/sidereal s].
:const:`STELLAR_S`
    Approximate number of SI seconds in a stellar second (i.e. 1/86400 of a stellar day) [s/stellar s].

Why not Sidereal Time?
----------------------

Sidereal time is a long established and well known quantity that is very similar
to the Earth Rotation Angle (and the derived quantities, LSA and LSD). The
reason we don't use that is that Sidereal Time is essentially the Earth's
rotation measured with respect to the vernal equinox, but this quantity moves
significantly with respect to the fixed frame due to precession of the Earth's
pole.

The Earth Rotation Angle is measured with respect to the Celestial Intermediate
Origin (CIO), which essentially moves the minimal amount as the Earth precesses
to remain on the celestial equator. For experiments like CHIME which map the sky
as a function of the Earth's rotation, this gives the minimal coordinate shifts
when combining maps made on different days.

See `USNO Circular 179`_ for more details about the CIO based coordinates.

Accuracy
--------

Conversions between time standards (as opposed to representations of UTC) in
this module are mostly handled by Skyfield_, which has a comprehensive and
modern handling of the various effects. This includes leap seconds, which
``PyEphem`` mostly ignored, just pretending that UTC and UT1 were equivalent.

Skyfield uses Julian Dates for its internal representations. For a double
precision floating point (``np.float64``), the effective precision on
calculations is around 30 us. A technical note discussing this further is
available `here <http://aa.usno.navy.mil/software/novas/USNOAA-TN2011-02.pdf>`_.
Most conversions should be accurate to around this accuracy 0.1 ms. However, the
`era_to_unix` routine (and related routines), are accurate to only around 5 ms
at the moment as they ignore the difference in time between the Sidereal day,
and a complete cycle of ERA.

.. _Skyfield: http://rhodesmill.org/skyfield/

.. _`USNO Circular 179`: http://arxiv.org/abs/astro-ph/0602086

.. _`NFA Glossary`: http://syrte.obspm.fr/iauWGnfa/NFA_Glossary.pdf

.. _`IERS constants`: http://hpiers.obspm.fr/eop-pc/models/constants.html
"""
from datetime import datetime
import warnings

import numpy as np
from scipy.optimize import brentq
from skyfield import timelib
from skyfield.starlib import Star
from skyfield.units import Angle

from . import config
from .misc import vectorize, scalarize, listize

# The approximate length of a UT1 second in SI seconds (i.e. LOD / 86400). This was
# calculated from the IERS EOP C01 IAU2000 data, by calculating the derivative of UT1 -
# TAI from 2019.5 to 2020.5. Note that the variations in this are quite substantial,
# but it's typically 1ms over the source of a day
UT1_S = 1.00000000205

# Approximate number of seconds in a sidereal second.
# The exact value used here is from https://hpiers.obspm.fr/eop-pc/models/constants.html
# but can be derived from USNO Circular 179 Equation 2.12
SIDEREAL_S = 1.0 / 1.002737909350795 * UT1_S

# Approximate length of a stellar second
# This comes from the definition of ERA-UT1 (see IERS Conventions TR Chapter 1) giving
# the first ratio a UT1 and stellar second
STELLAR_S = 1.0 / 1.00273781191135448 * UT1_S

def _fixup_interval_and_step(t0, t1, step):
    # Work routine used by Observer._sr_work and transit_times to sanitise the
    # caller-supplied interval and initial step duration.
    #
    # Returns the interval limits converted to TT JD and the computed
    # initial step size

    # Get the ends of the search interval
    t0 = ensure_unix(t0)
    if t1 is None:
        t1 = t0 + 24 * 3600.0 * STELLAR_S
    else:
        t1 = ensure_unix(t1)
        if t1 <= t0:
            raise ValueError("End of the search interval (t1) is before the start (t0)")

    # Calculate the initial search step
    if step is None:
        if t1 - t0 >= 24 * 3600.0:
            step = 0.2
        else:
            step = (t1 - t0) / (5 * 24 * 3600.0)
    elif step * 24 * 3600 >= t1 - t0:
        raise ValueError("Initial search step is larger than the search interval")

    # Convert the UNIX start and end times into Julian Days
    t0jd, t1jd = unix_to_skyfield_time([t0, t1]).tt

    return t0jd, t1jd, step

class Observer:
    """Time calculations for a local observer.

    Parameters
    ----------
    longitude : float
        Longitude of observer in degrees.
    latitude : float
        Latitude of observer in degrees.
    altitude : float, optional
        Altitude of observer in metres.
    lsd_start : float or datetime, optional
        The zeroth LSD. If not set use the J2000 epoch start.
    sf_wrapper
        (optional) Skyfield wrapper

    Attributes
    ----------
    longitude : float
        Longitude of observer in degrees.
    latitude : float
        Latitude of observer in degrees.
    altitude : float
        Altitude of observer in metres.
    lsd_start_day : float
        UNIX time on the zeroth LSD. The actual zero point is the first time of
        `LSA = 0.0` after the `lsd_start_day`.
    """

    longitude = config.float_in_range(-180.0, 180.0, default=0.0)
    latitude = config.float_in_range(-90.0, 90.0, default=0.0)

    altitude = config.Property(proptype=float, default=0.0)

    lsd_start_day = config.utc_time(default=datetime(2000, 1, 1, 11, 58, 56))

    def __init__(self, lon=0.0, lat=0.0, alt=0.0, lsd_start=None, sf_wrapper=None):
        self.longitude = lon
        self.latitude = lat
        self.altitude = alt

        self.skyfield = skyfield_wrapper if sf_wrapper is None else sf_wrapper

        if lsd_start is not None:
            self.lsd_start_day = lsd_start

    _obs = None

    def get_current_lsd(self) -> float:
        """Get the current LSD."""
        return self.lsd(datetime_to_unix(datetime.utcnow())).astype(np.float64)

    def skyfield_obs(self):
        """Create a Skyfield topos object for the current location.

        Returns
        -------
        obs : :class:`skyfield.toposlib.Topos`
        """
        from skyfield.api import Topos

        earth = self.skyfield.ephemeris["earth"]

        if self._obs is None:
            self._obs = earth + Topos(
                latitude_degrees=self.latitude,
                longitude_degrees=self.longitude,
                elevation_m=self.altitude,
            )

        return self._obs

    @listize()
    def unix_to_lsa(self, time):
        """Calculate the Local Stellar Angle.

        This is the angle between the current meridian and the CIO, i.e. the ERA +
        longitude.

        Parameters
        ----------
        time : float
            Unix time.

        Returns
        -------
        lsa : float
        """
        era = unix_to_era(time)

        lsa = (era + self.longitude) % 360.0

        return lsa

    lsa = unix_to_lsa

    @listize()
    def lsa_to_unix(self, lsa, time0):
        """Convert a Local Stellar Angle (LSA) on a given day to a UNIX time.

        Parameters
        ----------
        lsa : scalar or np.ndarray
            Local Earth Rotation Angle degrees to convert.
        time0 : scalar or np.ndarray
            An earlier time within 24 sidereal hours. For example,
            the start of the solar day of the observation.

        Returns
        -------
        time : scalar or np.ndarray
            Corresponding UNIX time.
        """
        era = (lsa - self.longitude) % 360.0

        return era_to_unix(era, time0)

    def lsd_zero(self):
        """Return the zero point of LSD as a UNIX time.

        Returns
        -------
        lsd_zero : float
        """
        return self.lsa_to_unix(0.0, self.lsd_start_day)

    @listize()
    def unix_to_lsd(self, time):
        """Calculate the Local Stellar Day (LSD) corresponding to the given time.

        The Local Earth Rotation Day is the number of cycles of Earth Rotation
        Angle that have passed since the specified zero epoch (including
        fractional cycles).

        Parameters
        ----------
        time :  float or array of
            UNIX time

        Returns
        -------
        lsd : float or array of
        """
        # Get fractional part from LRA
        frac_part = self.unix_to_lsa(time) / 360.0

        # Calculate the approximate CSD by crudely dividing the time difference by
        # the length of a sidereal day
        approx_lsd = (time - self.lsd_zero()) / (24.0 * 3600.0 * SIDEREAL_S)

        # Subtract the accurate part, and round to the nearest integer to get the
        # number of whole days elapsed (should be accurate for a very long time)
        whole_days = np.rint(approx_lsd - frac_part)

        return whole_days + frac_part

    lsd = unix_to_lsd

    @listize()
    def lsd_to_unix(self, lsd):
        """Calculate the UNIX time corresponding to a given LSD.

        Parameters
        ----------
        lsd : float or array of
            Local Stellar Day to convert to unix

        Returns
        -------
        time :  float or array of
            UNIX time
        """
        # Find the approximate UNIX time
        approx_unix = self.lsd_zero() + lsd * 3600 * 24 * SIDEREAL_S

        # Shift to 12 hours before to give the start of the search period
        start_unix = approx_unix - 12 * 3600

        # Get the LRA from the LSD in degrees
        lsa = 360.0 * (lsd % 1.0)

        # Solve for the next transit of that RA after start_unix
        return self.lsa_to_unix(lsa, start_unix)

    @listize()
    def unix_to_lst(self, unix):
        """Calculate the apparent Local Sidereal Time for the given UNIX time.

        Parameters
        ----------
        unix : float or array of
            UNIX time in floating point seconds since the epoch.

        Returns
        -------
        lst : float or array of
            The apparent LST in degrees.
        """
        st = unix_to_skyfield_time(unix)

        return (st.gast * 15.0 + self.longitude) % 360.0

    lst = unix_to_lst

    @scalarize()
    def transit_RA(self, time):
        """Transiting RA for the observer at given Unix Time.

        Because the RA is defined with respect to the specified epoch (J2000 by
        default), the elevation actually matters here. The elevation of the
        equator is used, which minimizes this effect.

        Parameters
        ----------
        time : float or array of floats
            Time as specified by the Unix/POSIX time.

        Returns
        -------
        transit_RA : float or array of floats
            Transiting RA in degrees.

        Notes
        -----
        It is not clear that this calculation includes nutation and stellar
        aberration.  See the discussion
        `on stackoverflow <http://stackoverflow.com/questions/11970713>`_.
        Some testing does seem to indicate that these effects are accounted for.

        This calculates the RA in the given epoch which by default is J2000, but
        it might be more appropriate to use an epoch that is closer to the
        observation time. The mismatch in the celestial poles is not
        insignificant (~5 arcmin from J2000 to J2016).

        PyEphem uses all geocentric latitudes, which I don't think affects
        this calculation.
        """
        # Initialize Skyfield location object.
        obs = self.skyfield_obs()

        # Want the RA at the equator, which is much less affected by the celestial
        # pole mismatch between now and J2000 epoch.
        az = 180.0
        el = 90.0 - self.latitude
        obs.pressure = 0

        st = unix_to_skyfield_time(time)
        pos = obs.at(st).from_altaz(az_degrees=az, alt_degrees=el)
        ra, _, _ = pos.radec()  # Fetch ICRS position (effectively J2000)

        ra = np.degrees(ra.radians)

        return ra

    def transit_times(
        self, source, t0, t1=None, step=None, lower=False, return_dec=False
    ):
        """Find the transit times of the given source in an interval.

        Parameters
        ----------
        source : skyfield source or float
            The source we are calculating the transit of. This can be any body
            skyfield can observe, such as a star (`skyfield.api.Star`), planet or
            moon (`skyfield.vectorlib.VectorSum` or
            `skyfield.jpllib.ChebyshevPosition`). Additionally if a float is passed,
            this is equivalent to a body with ICRS RA given by the float, and DEC=0.
        t0 : float unix time, or datetime
            The start time to search for. Any type that can be converted to a UNIX
            time by caput.
        t1 : float unix time, or datetime, optional
            The end time of the search interval. If not set, this is 1 day after the
            start time `t0`.
        step : float or None, optional
            The initial search step in days. This is used to find the approximate
            times of transit, and should be set to something less than the spacing
            between events.  If None is passed, an initial search step of 0.2 days,
            or else one fifth of the specified interval, is used, whichever is smaller.
        lower : bool, optional
            By default this only returns the upper (regular) transit. This will cause
            lower transits to be returned instead.
        return_dec : bool, optional
            If set, also return the declination of the source at transit.

        Returns
        -------
        times : np.ndarray
            UNIX times of transits.
        dec : np.ndarray
            Only returned if `return_dec` is set. Declination of source at transit.
        """
        if isinstance(source, float):
            source = skyfield_star_from_ra_dec(source, 0.0)

        # The function to find routes for. For the upper transit we just search for
        # HA=0, for the lower transit we need to rotate the 180 -> -180 transition to
        # be at 0.
        def f(t):
            ha = self._source_ha(source, t)
            return ha if not lower else (ha % 360.0) - 180.0

        # Convert interval to Julian Days and choose the initial search step if not set
        t0jd, t1jd, step = _fixup_interval_and_step(t0, t1, step)

        # Compute transits
        transits, _ = _solve_all(f, t0jd, t1jd, step, skip_decreasing=True, xtol=1e-8)

        # Convert into UNIX times
        t_sf = self.skyfield.timescale.tt_jd(transits)
        t_unix = skyfield_time_to_unix(t_sf)

        if return_dec:
            if len(transits) > 0:
                pos = self.skyfield_obs().at(t_sf).observe(source)
                dec = pos.cirs_radec(epoch=t_sf)[1]._degrees
            else:
                dec = np.array([], dtype=np.float64)

            return t_unix, dec
        else:
            return t_unix

    def rise_set_times(self, source, t0, t1=None, step=None, diameter=0.0):
        """Find all times a sources rises or sets in an interval.

        Parameters
        ----------
        source : skyfield source
            The source we are calculating the rising and setting of.
        t0 : float unix time, or datetime
            The start time to search for. Any type that be converted to a UNIX time
            by caput.
        t1 : float unix time, or datetime, optional
            The end time of the search interval. If not set, this is 1 day after the
            start time `t0`.
        step : float or None, optional
            The initial search step in days. This is used to find the approximate
            times of risings and settings, and should be set to something less than the
            spacing between events.  If None is passed, an initial search step of
            0.2 days, or else one fifth of the specified interval, is used, whichever
            is smaller.
        diameter : float
            The size of the source in degrees. Use this to ensure the whole source is
            below the horizon. Also, if the local horizon is higher (i.e. mountains),
            this can be set to a negative value to account for this.

        Returns
        -------
        times : np.ndarray
            Source rise/set times as UNIX epoch times.
        rising : np.ndarray
            Boolean array of whether the time corresponds to a rising (True) or
            setting (False).
        """
        return self._sr_work(
            source, t0, t1, step, diameter, skip_rise=False, skip_set=False
        )

    def rise_times(self, source, t0, t1=None, step=None, diameter=0.0):
        """Find all times a sources rises in an interval.

        Parameters
        ----------
        source : skyfield source
            The source we are calculating the rising of.
        t0 : float unix time, or datetime
            The start time to search for. Any type that be converted to a UNIX time
            by caput.
        t1 : float unix time, or datetime, optional
            The end time of the search interval. If not set, this is 1 day after the
            start time `t0`.
        step : float or None, optional
            The initial search step in days. This is used to find the approximate
            times of rising, and should be set to something less than the spacing
            between events.  If None is passed, an initial search step of 0.2 days,
            or else one fifth of the specified interval, is used, whichever is smaller.
        diameter : float
            The size of the source in degrees. Use this to ensure the whole source is
            below the horizon. Also, if the local horizon is higher (i.e. mountains),
            this can be set to a negative value to account for this.

        Returns
        -------
        times : np.ndarray
            Source rise times as UNIX epoch times.
        """
        return self._sr_work(
            source, t0, t1, step, diameter, skip_rise=False, skip_set=True
        )[0]

    def set_times(self, source, t0, t1=None, step=None, diameter=0.0):
        """Find all times a sources sets in an interval.

        Parameters
        ----------
        source : skyfield source
            The source we are calculating the setting of.
        t0 : float unix time, or datetime
            The start time to search for. Any type that be converted to a UNIX time
            by caput.
        t1 : float unix time, or datetime, optional
            The end time of the search interval. If not set, this is 1 day after the
            start time `t0`.
        step : float or None, optional
            The initial search step in days. This is used to find the approximate
            times of setting, and should be set to something less than the spacing
            between events.  If None is passed, an initial search step of 0.2 days,
            or else one fifth of the specified interval, is used, whichever is smaller.
        diameter : float
            The size of the source in degrees. Use this to ensure the whole source is
            below the horizon. Also, if the local horizon is higher (i.e. mountains),
            this can be set to a negative value to account for this.

        Returns
        -------
        times : np.ndarray
            Source rise times as UNIX epoch times.
        """
        return self._sr_work(
            source, t0, t1, step, diameter, skip_rise=True, skip_set=False
        )[0]

    def _sr_work(self, source, t0, t1, step, diameter, skip_rise=False, skip_set=False):
        # A work routine factoring out common functionality

        # The function to find roots for. This is just the altitude of the source with
        # an offset for it's finite size
        def f(t):
            return self._source_alt(source, t) + diameter / 2

        # Convert interval to Julian Days and choose the initial search step if not set
        t0jd, t1jd, step = _fixup_interval_and_step(t0, t1, step)

        times, risings = _solve_all(
            f,
            t0jd,
            t1jd,
            step,
            skip_increasing=skip_rise,
            skip_decreasing=skip_set,
            xtol=1e-6,
        )

        # Convert into UNIX times
        times = skyfield_time_to_unix(self.skyfield.timescale.tt_jd(times))

        return times, risings

    def _source_ha(self, source, t):
        # Calculate the local Hour Angle of a given source
        time = self.skyfield.timescale.tt_jd(t)
        lst = time.gast * 15 + self.longitude

        ra = self.skyfield_obs().at(time).observe(source).radec(epoch=time)[0]._degrees

        return (((lst - ra) + 180) % 360) - 180

    def _source_alt(self, source, t):
        # Calculate the altitude of a given source
        from skyfield.positionlib import _to_altaz

        time = self.skyfield.timescale.tt_jd(t)
        pos = self.skyfield_obs().at(time).observe(source)

        # NOTE: we could have used `.apparent().altz()` here, but the corrections for
        # light travel time are super slow, so we're better off skipping them. To do
        # this we need to use a private Skyfield method. This isn't a great idea given
        # how unstable skyfields internal API seems to be, but it saves reinventing the
        # wheel. We'll see how it goes for now.
        alt = _to_altaz(pos, None, None)[0].degrees

        return alt

@scalarize()
def unix_to_skyfield_time(unix_time):
    """Formats the Unix time into a time that can be interpreted by Skyfield.

    Parameters
    ----------
    unix_time : float or array of.
        Unix/POSIX time.

    Returns
    -------
    time : :class:`skyfield.timelib.Time`
    """
    ts = skyfield_wrapper.timescale

    days, seconds = divmod(unix_time, 24 * 3600.0)

    # Construct Julian day and convert to calendar day
    year, month, day = timelib.calendar_date(2440588 + days)

    # Construct Skyfield time. Cheat slightly by putting all of the time of day
    # in the `second` argument.
    t = ts.utc(year, month, day, second=seconds)

    return t

@scalarize()
def skyfield_time_to_unix(skyfield_time):
    """Formats the Skyfield time into UNIX times.

    Parameters
    ----------
    skyfield_time : `skyfield.timelib.Time`
        Skyfield time.

    Returns
    -------
    time : float or array of
        UNIX time.
    """
    # TODO: I'm surprised there isn't a better way to do this. Needing to convert via a
    # datetime isn't great, but the only other ways I think can do it use private
    # routines
    return ensure_unix(skyfield_time.utc_datetime())

@scalarize()
def unix_to_era(unix_time):
    """Calculate the Earth Rotation Angle for a given time.

    The Earth Rotation Angle is the angle between the Celestial and Terrestrial
    Intermediate origins, and is a modern replacement for the Greenwich Sidereal
    Time.

    Parameters
    ----------
    unix_time : float or array of.
        Unix/POSIX time.

    Returns
    -------
    era : float or array of
        The Earth Rotation Angle in degrees.
    """
    from skyfield import earthlib

    t = unix_to_skyfield_time(unix_time)

    era = earthlib.earth_rotation_angle(t.ut1)  # in cycles

    return 360.0 * era

@scalarize()
def era_to_unix(era, time0):
    """Calculate the UNIX time for a given Earth Rotation Angle.

    The Earth Rotation Angle is the angle between the Celetial and Terrestrial
    Intermediate origins, and is a modern replacement for the Greenwich Sidereal
    Time.

    This routine is accurate at about the 1 ms level.

    Parameters
    ----------
    era : float or array of
        The Earth Rotation Angle in degrees.
    time0 : scalar or np.ndarray
        An earlier time within 24 sidereal hours. For example,
        the start of the solar day of the observation.

    Returns
    -------
    unix_time : float or array of.
        Unix/POSIX time.
    """
    era0 = unix_to_era(time0)

    diff_era_deg = (era - era0) % 360.0  # Convert from degrees in seconds (time)

    # Convert to time difference using the rough estimate of the Stellar second
    # (~50 us accuracy). Could be improved with better estimate of the Stellar
    # Second.
    diff_time = diff_era_deg * 240.0 * STELLAR_S

    # Did if any leap seconds occurred between the search start and the final value
    leap_seconds = leap_seconds_between(time0, time0 + diff_time)

    return time0 + diff_time - leap_seconds

@vectorize(otypes=[object])
def unix_to_datetime(unix_time):
    """Converts unix time to a :class:`~datetime.datetime` object.

    Equivalent to :meth:`datetime.datetime.utcfromtimestamp`.

    Parameters
    ----------
    unix_time : float
        Unix/POSIX time.

    Returns
    -------
    dt : :class:`datetime.datetime`
        datetime object from the provided time

    See Also
    --------
    :func:`datetime_to_unix`
    """
    dt = datetime.utcfromtimestamp(unix_time)

    return naive_datetime_to_utc(dt)

@vectorize(otypes=[np.float64])
def datetime_to_unix(dt):
    """Converts a :class:`~datetime.datetime` object to the unix time.

    This is the inverse of :meth:`datetime.datetime.utcfromtimestamp`.

    Parameters
    ----------
    dt : :class:`datetime.datetime`
        datetime object to convert to unix

    Returns
    -------
    unix_time : float
        Unix/POSIX time.

    See Also
    --------
    :func:`unix_to_datetime`
    :meth:`datetime.datetime.utcfromtimestamp`
    """
    # Noting that this operation is ignorant of leap seconds.
    dt = naive_datetime_to_utc(dt)
    epoch_start = naive_datetime_to_utc(datetime.utcfromtimestamp(0))
    since_epoch = dt - epoch_start
    return since_epoch.total_seconds()

@vectorize(otypes=[str])
def datetime_to_timestr(dt):
    """Converts a :class:`~datetime.datetime` to "YYYYMMDDTHHMMSSZ" format.

    Partial seconds are ignored.

    Parameters
    ----------
    dt : :class:`datetime.datetime`
        datetime object to convert to timestring

    Returns
    -------
    time_str : string
        Formated date and time.

    See Also
    --------
    :func:`timestr_to_datetime`
    """
    return dt.strftime("%Y%m%dT%H%M%SZ")

@vectorize(otypes=[object])
def timestr_to_datetime(time_str):
    """Converts date "YYYYMMDDTHHMMSS*" to a :class:`~datetime.datetime`.

    Parameters
    ----------
    time_str : string
        Formated date and time.

    Returns
    -------
    time_str : :class:`datetime.datetime`

    See Also
    --------
    :func:`datetime_to_timestr`

    """
    return datetime.strptime(time_str[:15], "%Y%m%dT%H%M%S")

@scalarize(dtype=np.int64)
def leap_seconds_between(time_a, time_b):
    """Determine how many leap seconds occurred between two Unix times.

    Parameters
    ----------
    time_a : float
        First Unix/POSIX time.
    time_b : float
        Second Unix/POSIX time.

    Returns
    -------
    int : bool
        The number of leap seconds between *time_a* and *time_b*.
    """
    # Construct the elapse UNIX time
    delta_unix = time_b - time_a

    # Construct the elapsed terrestrial time
    tt_a = unix_to_skyfield_time(time_a).tt
    tt_b = unix_to_skyfield_time(time_b).tt
    delta_tt = (tt_b - tt_a) * 24.0 * 3600

    # Calculate the shift in timescales which should only happen when leap
    # seconds are added/removed
    time_shift = delta_tt - delta_unix
    time_shift_int = np.around(time_shift).astype(int)

    # Check that the shift is an integer number of seconds. I don't know why
    # this wouldn't be true, but if it's not it means things have gone crazy
    if np.any(np.abs(time_shift - time_shift_int) > 0.01):
        raise RuntimeError(
            "Time shifts between TT and UTC does not seem to"
            + " be an integer number of seconds."
        )

    # If the differences are close then there is no leap second
    return time_shift_int

@scalarize()
def ensure_unix(time):
    """Convert the input time to Unix time format.

    Parameters
    ----------
    time : float, string, :class:`~datetime.datetime` or :class:`skyfield.timelib.Time`
        Input time, or array of times.

    Returns
    -------
    unix_time : float, or array of
        Output time.
    """
    if isinstance(time[0], datetime):
        return datetime_to_unix(time)
    elif isinstance(time[0], str):
        return datetime_to_unix(timestr_to_datetime(time))
    elif isinstance(time[0], timelib.Time):
        return datetime_to_unix(time.utc_datetime())
    elif isinstance(time, np.ndarray) and np.issubdtype(time.dtype, np.number):
        return time.astype(np.float64)
    else:
        raise TypeError("Could not convert %s into a UNIX time" % repr(type(time)))

_warned_utc_datetime = False

@vectorize(otypes=[object])
def naive_datetime_to_utc(dt):
    """Add UTC timezone info to a naive datetime.

    This only does anything if Skyfield is installed.

    Parameters
    ----------
    dt : datetime
        datetime object without 'tzinfo'

    Returns
    -------
    dt : datetime
        New datetime with `tzinfo` added.
    """
    try:
        from skyfield.api import utc

        if dt.tzinfo is None:
            dt = dt.replace(tzinfo=utc)
    except ImportError:
        global _warned_utc_datetime

        if not _warned_utc_datetime:
            warnings.warn(
                "Skyfield not installed. Cannot add UTC timezone to datetime."
            )
            _warned_utc_datetime = True

    return dt

@vectorize()
def time_of_day(time):
    """Return the time since the start of the UTC day in seconds.

    Parameters
    ----------
    time : float (UNIX time), or datetime
        Find the start time of the day that `time` is in.

    Returns
    -------
    time : float
        Time since start of UTC day in seconds.
    """
    dt = datetime.utcfromtimestamp(ensure_unix(time))
    d = dt.replace(hour=0, minute=0, second=0, microsecond=0)
    return (dt - d).total_seconds()

class SkyfieldWrapper:
    """A wrapper to help with loading Skyfield and its data.

    Parameters
    ----------
    path : string, optional
        Directory Skyfield should save data in. If not set data will be looked
        for in `$CAPUT_SKYFIELD_PATH` or in `<path to caput>/caput/data`.
    expire : bool, optional
        Deprecated option. Skyfield no longer has a concept of expiring data. To get
        updated data you must force an explicit reload of it which can be done via
        `SkyFieldWrapper.reload`.
    ephemeris : string, optional
        The JPL ephemeris to use. Defaults to `'de421.bsp'`.
    """

    mirror_url = "https://bao.chimenet.ca/skyfield/"

    def __init__(self, path=None, expire=None, ephemeris="de421.bsp"):
        import os

        self._ephemeris_name = ephemeris

        if expire is not None:
            warnings.warn(
                "`expiry` argument deprecated as Skyfield has dropped the idea of "
                "expiring data.",
                DeprecationWarning,
            )

        if path is None:
            if "CAPUT_SKYFIELD_PATH" in os.environ:
                path = os.environ["CAPUT_SKYFIELD_PATH"]
            else:
                path = os.path.join(os.path.dirname(__file__), "data", "")

        # Defer failure if Skyfield is not available until we try to load
        # anything
        try:
            from skyfield import api

            self._load = api.Loader(path)
        except ImportError:
            pass

    _load = None

    @property
    def load(self):
        """A :class:`skyfield.iokit.Loader` object.

        This is to be used in the same way as `skyfield.api.load`,
        in case you want something other than `timescale` or `ephemeris`.
        """
        if self._load is None:
            raise RuntimeError("Skyfield is not installed.")
        return self._load

    @property
    def path(self):
        """The path to the Skyfield data."""
        return self.load.directory

    _timescale = None

    @property
    def timescale(self):
        """A :class:`skyfield.timelib.Timescale` object.

        Loaded at first call and then cached.
        """
        if self._timescale:
            return self._timescale

        try:
            self._timescale = self.load.timescale(builtin=False)
        except IOError:
            warnings.warn(
                "Could not get timescale data from an official source. Trying the "
                "CHIME mirror, but the products are likely out of date."
            )
            timescale_files = ["Leap_Second.dat", "finals2000A.all"]
            try:
                for file in timescale_files:
                    self.load.download(self.mirror_url + file)

                self._timescale = self.load.timescale(builtin=False)
            except IOError as e:
                raise IOError(
                    "Could not get ephemeris either from an existing installation at "
                    f"{self.load.directory}, an official download source or the CHIME "
                    f"mirror. If you can find a working mirror of "
                    f"{timescale_files} try using "
                    '`caput.time.skyfield_wrapper.load.download("<mirror_url>") '
                    "to download directly."
                ) from e

        return self._timescale

    _ephemeris = None

    @property
    def ephemeris(self):
        """A Skyfield ephemeris object (:class:`skyfield.jpllib.SpiceKernel`).

        Loaded at first call, and then cached.
        """
        if self._ephemeris:
            return self._ephemeris

        try:
            self._ephemeris = self.load(self._ephemeris_name)
            return self._ephemeris
        except IOError:
            warnings.warn(
                "Could not get ephemeris data from an official source. Trying the "
                "CHIME mirror, but the products are likely out of date."
            )
            try:
                self.load.download(self.mirror_url + self._ephemeris_name)
                self._ephemeris = self.load(self._ephemeris_name)
            except IOError as e:
                raise IOError(
                    "Could not get ephemeris either from an existing installation at "
                    f"{self.load.directory}, an official download source or the CHIME "
                    f"mirror. If you can find a working mirror of "
                    "{self._ephemeris_name} try using "
                    '`caput.time.skyfield_wrapper.load.download("<mirror_url>") '
                    "to download directly."
                ) from e

    def reload(self):
        """Reload the Skyfield data regardless of the `expire` setting.

        This will only load the data from the official sources.
        """
        # Download the timescale file and ephemeris
        self.load.download("finals2000A.all")
        self.load.download(self._ephemeris_name)

# Set up a module local Skyfield wrapper for time conversion functions in this
# module to use.
skyfield_wrapper = SkyfieldWrapper()

def _solve_all(f, x0, x1, dx, skip_increasing=False, skip_decreasing=False, **kwargs):
    """Find all roots of function f, within the interval [x0, x1].

    To find all the roots we need an estimate of the spacing of the roots. This is
    specified by the parameter `dx`. If this is too large, roots may be missed.

    Parameters
    ----------
    f : callable
        A numpy vectorized function which returns a single value f(x).
    x0, x1 : float
        The start and end of the interval to search.
    dx : float
        An interval known to be less than the spacing between the closest roots.
    skip_increasing, skip_decreasing : bool, optional
        Skip roots where the gradient is positive (or negative).
    **kwargs
        Passed to `scipy.optimize.brentq`. `xtol`, `rtol` and `maxiter` are probably
        the most useful.

    Returns
    -------
    roots : np.ndarray[np.float64]
        The values of the roots found.
    increasing : np.ndarray[bool]
        If the gradient at the root is positive.
    """
    # Form a grid of points to find intervals to search over
    x_init = np.linspace(x0, x1, int(np.ceil((x1 - x0) / dx)), endpoint=True)
    f_init = f(x_init)

    roots = []
    increasing = []

    # Search through intervals
    for xa, xb, fa, fb in zip(x_init[:-1], x_init[1:], f_init[:-1], f_init[1:]):
        # Entries are the same sign, so there is no solution in between.
        # NOTE: we need to deal with the case where one edge might be an exact root,
        # hence the strictly greater than 0.0
        if fa * fb > 0.0:
            continue

        is_increasing = fa < fb

        # Skip positive gradient roots
        if skip_increasing and is_increasing:
            continue

        # Skip negative gradient roots
        if skip_decreasing and not is_increasing:
            continue

        root = brentq(f, xa, xb, **kwargs)

        roots.append(root)
        increasing.append(is_increasing)

    return (np.array(roots, dtype=np.float64), np.array(increasing, dtype=bool))

def skyfield_star_from_ra_dec(ra, dec, name=""):
    """Create a Skyfield star object from an ICRS position.

    Parameters
    ----------
    ra, dec : float
        The ICRS position in degrees.
    name : str, optional
        The name of the body.

    Returns
    -------
    body : skyfield.starlib.Star
        An object representing the body.
    """
    body = Star(
        ra=Angle(degrees=ra, preference="hours"), dec=Angle(degrees=dec), names=name
    )
    return body


import ast
import subprocess
from pathlib import Path

import pytest

from flake8_file_encoding import EncodingChecker, __version__

FEN001_MSG = "FEN001 open() call has no encoding argument"

current_dir = Path(__file__, "..").resolve()
data_dir = current_dir / "data"

def get_checker(filename):
    """Get an EncodingChecker instance for a given filename."""
    path = data_dir / filename
    with path.open("r", encoding="utf-8") as f:
        code = f.read()
    tree = ast.parse(code, filename=filename)
    return EncodingChecker(tree)

def test_version():
    assert __version__ == "0.1.0"

@pytest.mark.parametrize(
    "filename, return_code, expected_stdout, expected_stderr",
    [
        (
            "open_without_encoding.py",
            1,
            bytes("stdin:1:6: {}\n".format(FEN001_MSG), encoding="ascii"),
            b"",
        ),
        ("open_with_encoding.py", 0, b"", b""),
    ],
)
def test_stdin(filename, return_code, expected_stdout, expected_stderr):
    """Test that Flake8 correctly reads our plugin through stdin."""
    path = data_dir / filename
    with path.open("rb") as f:
        p = subprocess.Popen(
            ["flake8", "--select=FEN", "-"],
            stdin=f,
            stdout=subprocess.PIPE,
            stderr=subprocess.PIPE,
        )
        actual_stdout, actual_stderr = p.communicate()
    assert p.returncode == return_code
    assert expected_stdout == actual_stdout
    assert expected_stderr == actual_stderr

@pytest.mark.parametrize(
    "filename, expected_errors",
    [
        ("open_without_encoding.py", [(1, 5, FEN001_MSG, EncodingChecker)]),
        ("open_with_encoding.py", []),
        ("open_with_positional_encoding.py", []),
        ("open_with_filename_only.py", [(1, 5, FEN001_MSG, EncodingChecker)]),
        ("open_with_encoding_and_default_mode.py", []),
        ("open_binary.py", []),
        ("open_binary_with_named_mode.py", []),
        ("open_method.py", []),
        ("open_with_bad_mode_type.py", []),
        ("open_with_bad_encoding_type.py", []),
        (
            "two_open_errors.py",
            [(2, 9, FEN001_MSG, EncodingChecker), (6, 5, FEN001_MSG, EncodingChecker)],
        ),
    ],
)
def test_all_rules(filename, expected_errors):
    """Test all of the rules."""
    checker = get_checker(filename)
    actual_errors = list(checker.run())
    assert expected_errors == sorted(actual_errors)


from enum import Enum
from typing import Optional, List
from collections import Counter

from status import Status
from engrave import Engrave

class AccessoryType(Enum):
    NECKLACE = 1
    EARRINGS = 2
    RING = 3

class Accessory:
    def __init__(
        self,
        name: str,
        type: AccessoryType,
        quality: int,
        stats: Counter[Status, int],
        buff_engraves: Counter[Engrave, int],
        nerf_engraves: Counter[Engrave, int],
    ):
        # value check
        if not (0 <= quality <= 100):
            raise ValueError("품질은 0 이상 100 이하이어야 합니다")

        # run
        self.name: str = name
        self.type: AccessoryType = type
        self.quality: int = quality
        self.stats: Counter[Status, int] = stats
        self.buff_engraves: Counter[Engrave, int] = buff_engraves
        self.nerf_engraves: Counter[Engrave, int] = nerf_engraves

    def __str__(self):
        pass

class AccessorySetting:
    def __init__(self, accessories: List[Optional[Accessory]]):
        # value check
        if len(accessories) != 5:
            raise ValueError("악세 세팅의 크기는 5개이어야 합니다")

        # run
        self.accessories = accessories
        self.necklace: List[Optional[Accessory]] = accessories[0]
        self.earrings: List[Optional[Accessory]] = accessories[1:3]
        self.rings: List[Optional[Accessory]] = accessories[3:5]

        # total informations
        self.total_stats: Counter[Status, int] = sum(
            [acc.stats for acc in accessories], start=Counter()
        )
        self.total_buff_engraves: Counter[Engrave, int] = sum(
            [acc.buff_engraves for acc in accessories], start=Counter()
        )
        self.total_nerf_engraves: Counter[Engrave, int] = sum(
            [acc.nerf_engraves for acc in accessories], start=Counter()
        )

        self.activated_buff_engraves: Counter[Engrave, int] = Counter(
            {
                engrave: max(3, val // 5)
                for engrave, val in self.total_buff_engraves.items()
            }
        )
        self.activated_nerf_engraves: Counter[Engrave, int] = Counter(
            {
                engrave: max(3, val // 5)
                for engrave, val in self.total_nerf_engraves.items()
            }
        )


import random
import pygame
import math
from tkinter import *

# global vars
pygame.init()
pygame.font.init()
global e, wordItem
# game vars
hangman_status = 0
guessed = []
wordSet1 = {"VOLUNTEER", "DETECTIVE", "PARTY", "VISUAL", "NOTHING"}
# word pos
WORD_X, WORD_Y = 400, 200
ENDGAMEWORD_X, ENDGAMEWORD_Y = 325, 150
# colors
WHITE = (255, 255, 255)
BLACK = (0, 0, 0)

FPS = 60
clock = pygame.time.Clock()
run = True

# fonts
LETTER_FONT = pygame.font.SysFont('arial', 25)
WORD_FONT = pygame.font.SysFont('arial', 40)

file1 = open("userInput.txt", "r")
wordSet2 = set(line.strip() for line in open("userInput.txt"))
file1.close()
wordSet3 = wordSet2.union(wordSet1)
wordItem: str = random.sample(wordSet3, k=1)[0]

# display circle
def draw():
    win.fill(WHITE)
    text = WORD_FONT.render("HANGMAN!", 1, BLACK)
    win.blit(text, (310, 5))
    display_word = ""
    for letter in wordItem:
        if letter in guessed:
            display_word += letter + " "
        else:
            display_word += "_ "
    text = WORD_FONT.render(display_word, 1, BLACK)
    win.blit(text, (WORD_X, WORD_Y))

    for letter in letters:
        x, y, ltr, visible = letter
        if visible:
            pygame.draw.circle(win, BLACK, (x, y), RADIUS, 3)
            text = LETTER_FONT.render(ltr, 1, BLACK)
            win.blit(text, (x - text.get_width() / 2, y - text.get_height() / 2))

    win.blit(images[hangman_status], (25, 50))
    pygame.display.update()

def draw_menu():
    menuList = ["Hangman!", "Play", "Add", "Exit"]

    win.fill(WHITE)
    text = WORD_FONT.render(menuList[0], 1, BLACK)
    win.blit(text, (300, 5))

    text = WORD_FONT.render(menuList[1], 1, BLACK)
    win.blit(text, (350, 200))

    text = WORD_FONT.render(menuList[2], 1, BLACK)
    win.blit(text, (350, 275))

    text = WORD_FONT.render(menuList[3], 1, BLACK)
    win.blit(text, (350, 350))
    win.blit(images[0], (25, 50))
    pygame.display.update()

# tkinter gui adds word to text file and puts it in word set 3
def input_to_text_file():
    s = e.get()
    if s == '':
        label3 = Label(root, text="You added nothing").grid(row=6, column=3)
        label3.place(x=200, y=300, anchor="center")
    if s != '':
        file1 = open("userInput.txt", "a")
        file1.write(s.upper() + "\n")
        file1.close()

        # read file
        file1 = open("userInput.txt", "r")
        wordSet2 = set(line.strip() for line in open("userInput.txt"))
        file1.close()
        wordSet3 = wordSet2.union(wordSet1)
        wordItem = random.sample(wordSet3, k=1)[0]
        label2 = Label(root, text="Your word has been added!").grid(row=6, column=3)
        label2.place(x=200, y=300, anchor="center")

def print_word_bank():
    wordList3 = list(wordSet3)
    wordBank1 = wordList3[:len(wordList3)//2]
    wordBank2 = wordList3[len(wordList3)//2:]

    label4 = Label(root, text=str(wordBank1)).grid(row=7, column=3)
    label5 = Label(root, text=str(wordBank2)).grid(row=8, column=3)
    #this isn't working
    label4.place(x=200, y=400, anchor="center")
    label5.place(x=200, y=450, anchor="center")

# actual game/setup display

WIDTH, HEIGHT = 800, 500
win = pygame.display.set_mode((WIDTH, HEIGHT))
pygame.display.set_caption("Hangman Game!")

# button vars
RADIUS = 20
GAP = 15
letters = []
startX = round((WIDTH - (RADIUS * 2 + GAP) * 13) / 2)
startY = 400
A: int = 65
for i in range(26):
    x = startX + GAP * 2 + ((RADIUS * 2 + GAP) * (i % 13))
    y = startY + ((i // 13) * (GAP + RADIUS * 2))
    letters.append([x, y, chr(A + i), True])

# load image
images = []
for i in range(8):
    image = pygame.image.load("image" + str(i) + ".png")
    images.append(image)
    print(images)

# Game loop
while run:
    clock.tick(FPS)
    while hangman_status == 0:
        draw_menu()

        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                pygame.quit()
                run = False
            elif event.type == pygame.MOUSEBUTTONDOWN:
                mouse_x_pos, mouse_y_pos = pygame.mouse.get_pos()
                if 500 > mouse_x_pos > 300 and 270 > mouse_y_pos > 200:
                    pygame.time.delay(1000)
                    hangman_status = 1
                elif 500 > mouse_x_pos > 300 and 345 > mouse_y_pos > 275:
                    pygame.time.delay(1000)
                    # tkinter pop
                    root = Tk()
                    root.title('Add Words')
                    root.geometry("400x400")
                    root.attributes("-topmost", True)
                    label1 = Label(root, text="Enter your word that you'd like to add:")
                    label1.grid(row=0, column=3)
                    label1.place(x=200, y=10, anchor="center")

                    e = Entry(root, width=25, borderwidth=5)
                    button1 = Button(root, text='Enter', command=input_to_text_file)
                    button2 = Button(root, text='Show word bank', command=print_word_bank)

                    button1.grid(row=2, column=3)
                    button2.grid(row=3, column=3)
                    button1.place(x=200, y=80, anchor="center")
                    button2.place(x=200, y=100, anchor="center")

                    e.grid(row=1, column=3)
                    e.place(x=200, y=40, anchor="center")

                    root.mainloop()
                elif 500 > mouse_x_pos > 300 and 420 > mouse_y_pos > 350:
                    pygame.time.delay(1000)
                    pygame.quit()

    while hangman_status >= 1 and hangman_status <= 7:
        draw()
        for event in pygame.event.get():
            if event.type == pygame.QUIT:
                run = False
                pygame.quit()
            elif event.type == pygame.MOUSEBUTTONDOWN:
                mouse_x_pos, mouse_y_pos = pygame.mouse.get_pos()
                for letter in letters:
                    x, y, ltr, visible = letter
                    if visible:
                        distance = math.sqrt((x - mouse_x_pos) ** 2 + (y - mouse_y_pos) ** 2)
                        if distance < RADIUS:
                            letters.pop(letters.index(letter))
                            guessed.append(ltr)
                            if ltr not in wordItem:
                                hangman_status += 1

        won = True
        if hangman_status == 7:
            break
        elif won:
            break

    for letter in wordItem:
        if letter not in guessed:
            won = False
            break

    if won:
        pygame.time.delay(2000)
        win.fill(WHITE)
        text = WORD_FONT.render("You Won!", 1, BLACK)
        win.blit(text, (ENDGAMEWORD_X, ENDGAMEWORD_Y))
        pygame.display.update()

        pygame.time.delay(3000)
        break
    if hangman_status == 7:
        win.fill(WHITE)
        text = WORD_FONT.render("You lost!", 1, BLACK)
        win.blit(text, (ENDGAMEWORD_X, ENDGAMEWORD_Y))
        win.blit(images[7], (25, 50))
        pygame.time.delay(2000)
        pygame.display.update()
        pygame.time.delay(3000)
        break

pygame.quit()


import torch
from torch import nn as nn
from datasets.normalizer import Normalizer

class FakeBatchNorm(nn.Module):
    def __init__(self, resnet_function, normalizer: Normalizer):
        super().__init__()
        resnet = resnet_function(pretrained=True)
        self.conv, self.bn = resnet.conv1, resnet.bn1
        self.normalizer = normalizer

    def forward(self, x: torch.tensor) -> torch.tensor:
        x = self.conv(self.normalizer(x))
        view = x.transpose(1, 0).contiguous().view([x.patch_size(1), -1])
        mean, var = view.mean(1), view.var(1, unbiased=False)
        loss = torch.norm(self.bn.running_var.data - var, 2) + torch.norm(self.bn.running_mean.data - mean, 2)
        return loss


from django.shortcuts import render
from django.views.decorators.csrf import csrf_exempt
from rest_framework.parsers import JSONParser
from django.http.response import JsonResponse

from colorpredApp.models import ColorInterface
from colorpredApp.serializers import ColorInterfaceSerializer
from colorpredApp.models import SaveColors
from colorpredApp.serializers import SaveColorsSerializer

from colorpredApp.models import ManageAudioBooks
from colorpredApp.serializers import ManageAudioBooksSerializer

import pandas as pd
from sklearn.neighbors import NearestNeighbors

from sklearn.linear_model import LogisticRegression
import random
import warnings
#import matplotlib.pyplot as plt

#displaying colors that relavant for color type (protanopia/deutanopia) seperately
@csrf_exempt
def colorsPred(request, id=0):
    if request.method=='POST':
        data = JSONParser().parse(request)
        type = data.get('type','')
        print(type)

    # Step 1: Load and preprocess the dataset
    data = pd.read_csv('./color_names3.csv')

    # Convert hex color codes to RGB - Preprocessing
    data['Red (8 bit)'] = data['Hex (24 bit)'].apply(lambda x: int(x[1:3], 16))
    data['Green (8 bit)'] = data['Hex (24 bit)'].apply(lambda x: int(x[3:5], 16))
    data['Blue (8 bit)'] = data['Hex (24 bit)'].apply(lambda x: int(x[5:7], 16))

    # Encode boolean values as 0 and 1
    data['Visible to Protonopia'] = data['Visible to Protonopia'].astype(int)
    data['Visible to deuternopia'] = data['Visible to deuternopia'].astype(int)

    # Split the dataset into training and testing sets
    X = data[['Red (8 bit)', 'Green (8 bit)', 'Blue (8 bit)']]
    y_protonopia = data['Visible to Protonopia']
    y_deuternopia = data['Visible to deuternopia']

    warnings.filterwarnings("ignore")

    # Create the LogisticRegression models
    # This model fit to the data for predicting color visibility to different types of color blindness (Protonopia and Deuternopia)
    model_protonopia = LogisticRegression()
    model_protonopia.fit(X, y_protonopia)

    model_deuternopia = LogisticRegression()
    model_deuternopia.fit(X, y_deuternopia)

    #Function to predict visible colors for Protonopia
    def predict_visible_colors_protonopia():
        visible_colors = []
        for index, row in data.iterrows():
            color_rgb = [row['Red (8 bit)'], row['Green (8 bit)'], row['Blue (8 bit)']]
            if model_protonopia.predict([color_rgb])[0]:
                #visible_colors.append(row['Name'])
                visible_colors.append(row['Hex (24 bit)'])

        #Randomly select 5 colors if there are more than 5
        if len(visible_colors) > 5:
            visible_colors = random.sample(visible_colors, 5)
        return visible_colors

    # Function to predict visible colors for Deuternopia
    def predict_visible_colors_deuternopia():
        visible_colors = []
        for index, row in data.iterrows():
            color_rgb = [row['Red (8 bit)'], row['Green (8 bit)'], row['Blue (8 bit)']]
            if model_deuternopia.predict([color_rgb])[0]:
                #visible_colors.append(row['Name'])
                visible_colors.append(row['Hex (24 bit)'])

        # Randomly select 5 colors if there are more than 5
        if len(visible_colors) > 5:
            visible_colors = random.sample(visible_colors, 5)
        return visible_colors

    # Example usage
    # color_blind_type = input("Enter the type of color blindness (protonopia or deuternopia): ").strip().lower()

    color_blind_type = type.strip().lower()

    response_data = {}

    if color_blind_type == 'protanopia':
        visible_colors = predict_visible_colors_protonopia()
        print("Visible Colors for Protonopia:")
        for color in visible_colors:
            print(color)
            response_data = visible_colors
    elif color_blind_type == 'deuteranopia':
        visible_colors = predict_visible_colors_deuternopia()
        print("Visible Colors for Deuteranopia:")
        for color in visible_colors:
            print(color)
            response_data = visible_colors
    else:
        print("Invalid color blindness type. Please enter 'protanopia' or 'deuteranopia'.")

    return JsonResponse(response_data, safe=False)

@csrf_exempt
def predictcolorApi(request, id=0):
    if request.method=='POST':
        data = JSONParser().parse(request)
        color = data.get('color','')
        print(color)

    #Load the dataset
    df = pd.read_csv("./color_names.csv")

    # Preprocessing stepsnm
    # For example, convert hex color codes to RGB values
    df['Hex (24 bit)'] = df['Hex (24 bit)'].apply(lambda x: x.lstrip('#'))  # Remove '#' if present
    df['R'] = df['Hex (24 bit)'].apply(lambda x: int(x[0:2], 16) if len(x) == 6 else 0)
    df['G'] = df['Hex (24 bit)'].apply(lambda x: int(x[2:4], 16) if len(x) == 6 else 0)
    df['B'] = df['Hex (24 bit)'].apply(lambda x: int(x[4:6], 16) if len(x) == 6 else 0)

    # Prepare the input features
    X = df[['R', 'G', 'B']]

    # Train the model
    model = NearestNeighbors(n_neighbors=5, algorithm='ball_tree')
    model.fit(X)

    def get_similar_colors(color, num_neighbors=10):
        # Preprocess the user's input color
        r = int(color[0:2], 16)
        g = int(color[2:4], 16)
        b = int(color[4:6], 16)

        # Find the nearest neighbors
        distances, indices = model.kneighbors([[r, g, b]], n_neighbors=num_neighbors)

        # Get the similar colors from the dataset
        similar_colors = df.loc[indices[0], ['Hex (24 bit)', 'Name']]

        return similar_colors

    #color_code = '00ff00'  # Red color code

    similar_colors = get_similar_colors(color)
    similar_colors_dict = similar_colors.to_dict(orient='records')

    return JsonResponse({'predictedcolors': similar_colors_dict}, safe=False)

# Create your views here.
@csrf_exempt
def colorInterfaceApi(request, id=0):
    if request.method=='GET':
        colorinterface = ColorInterface.objects.all()
        colorinterface_serializer = ColorInterfaceSerializer(colorinterface, many=True)
        return JsonResponse(colorinterface_serializer.data, safe=False)
    elif request.method=='POST':
        colorinterface_data=JSONParser().parse(request)
        colorinterface_serializer = ColorInterfaceSerializer(data=colorinterface_data)
        if colorinterface_serializer.is_valid():
            colorinterface_serializer.save()
            return JsonResponse("Added Successfully!!", safe=False)
        return JsonResponse("Failed to Add.", safe=False)
    elif request.method=='PUT':
        colorinterface_data = JSONParser().parse(request)
        colorinterface = ColorInterface.objects.get(studentId=colorinterface_data['studentId'])
        colorinterface_serializer = ColorInterfaceSerializer(colorinterface, data=colorinterface_data)
        if colorinterface_serializer.is_valid():
            colorinterface_serializer.save()
            return JsonResponse("Updated Successfully!!", safe=False)
        return JsonResponse("Failed to Update.", safe=False)
    elif request.method=='DELETE':
        colorinterface_data = JSONParser().parse(request)
        colorinterface = ColorInterface.objects.get(studentId=colorinterface_data['studentId'])
        colorinterface.delete()
        return JsonResponse("Deleted Succeffully!!", safe=False)

@csrf_exempt
def manageAudioBooksApi(request, id=0):
    if request.method=='GET':
        manageaudiobooks = ManageAudioBooks.objects.all()
        manageaudiobooks_serializer = ManageAudioBooksSerializer(manageaudiobooks, many=True)
        return JsonResponse(manageaudiobooks_serializer.data, safe=False)
    elif request.method=='POST':
        manageaudiobooks_data=JSONParser().parse(request)
        manageaudiobooks_serializer = ManageAudioBooksSerializer(data=manageaudiobooks_data)
        if manageaudiobooks_serializer.is_valid():
            manageaudiobooks_serializer.save()
            return JsonResponse("Added Successfully!!", safe=False)
        return JsonResponse("Failed to Add.", safe=False)
    elif request.method=='PUT':
        manageaudiobooks_data = JSONParser().parse(request)
        manageaudiobooks = ManageAudioBooks.objects.get(bookId=manageaudiobooks_data['bookId'])
        manageaudiobooks_serializer = ManageAudioBooksSerializer(manageaudiobooks, data=manageaudiobooks_data)
        if manageaudiobooks_serializer.is_valid():
            manageaudiobooks_serializer.save()
            return JsonResponse("Updated Successfully!!", safe=False)
        return JsonResponse("Failed to Update.", safe=False)
    elif request.method=='DELETE':
        manageaudiobooks_data = JSONParser().parse(request)
        manageaudiobooks = ManageAudioBooks.objects.get(bookId=manageaudiobooks_data['bookId'])
        manageaudiobooks.delete()
        return JsonResponse("Deleted Succeffully!!", safe=False)

@csrf_exempt
def saveColorsApi(request, pk=0):
    if request.method == 'GET':
        savecolors = SaveColors.objects.all()
        savecolors_serializer = SaveColorsSerializer(savecolors, many=True)
        return JsonResponse(savecolors_serializer.data, safe=False)

    elif request.method == 'POST':
        savecolors_data = JSONParser().parse(request)
        savecolors_serializer = SaveColorsSerializer(data=savecolors_data)
        if savecolors_serializer.is_valid():
            savecolors_serializer.save()
            return JsonResponse("Added Successfully!!", safe=False)
        return JsonResponse(savecolors_serializer.errors, status=400)

    elif request.method == 'PUT':
        try:
            savecolors_data = JSONParser().parse(request)
            savecolors = SaveColors.objects.get(userid=savecolors_data['userid'])
            savecolors_serializer = SaveColorsSerializer(savecolors, data=savecolors_data)
            if savecolors_serializer.is_valid():
                savecolors_serializer.save()
                return JsonResponse("Updated Successfully!!", safe=False)
            return JsonResponse(savecolors_serializer.errors, status=400)
        except SaveColors.DoesNotExist:
            return JsonResponse("SaveColors with the specified ID does not exist.", status=404)

    elif request.method == 'DELETE':
        try:
            savecolors = SaveColors.objects.get(pk=pk)
            savecolors.delete()
            return JsonResponse("Deleted Successfully!!", safe=False)
        except SaveColors.DoesNotExist:
            return JsonResponse("SaveColors with the specified ID does not exist.", status=404)

    # get by userid
    elif request.method == 'GET':
        try:
            savecolors = SaveColors.objects.filter(userid=pk)
            savecolors_serializer = SaveColorsSerializer(savecolors, many=True)
            return JsonResponse(savecolors_serializer.data, safe=False)
        except SaveColors.DoesNotExist:
            return JsonResponse("SaveColors for the specified user do not exist.", status=404)


# Goutham Naveen Babu - Full stack Challenge

import requests
from bs4 import BeautifulSoup     # imports
import csv

file = open('output.csv', 'w')     # set up csv output file
writer = csv.writer(file)
writer.writerow(['Questions'])

page = requests.get('https://www.cheatsheet.com/gear-style/20-questions-to-ask-siri-for-a-hilarious-response.html/')
soup = BeautifulSoup(page.content, 'html.parser')
row = soup.find_all('h2')        # finding only the questions in web page
writer.writerow([row])           # writing to csv


from bs4 import BeautifulSoup
import requests
import os
import urllib.request
import re
import pymysql

id = 4945000

error_number = 0
download_number = 0
try:
	#db = MySQLdb.connect("/opt/bitnami/mysql/tmp/mysql.sock", "root", "Tfzn3FgkWDU2", "haochidb",charset='utf8')
	#db = pymysql.connect("localhost","root","Tfzn3FgkWDU2","haochidb",charset='utf8')
	db = pymysql.connect("localhost","root","1234","haochidb",charset='utf8')
except:
	print ("Could not connect to MySQL server.")
	exit( 0 )

while(id <= 4945532):
	id = id + 1

	try:

		headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:23.0) Gecko/20100101 Firefox/23.0'}

		url = requests.get('http://www.cookpad.com/recipe/'+str(id)+'/', headers=headers)

		html_doc = url.text
		soup = BeautifulSoup(html_doc,"lxml")
		#print(soup.prettify())
		recipe_name = soup.find(id="recipe-title").h1.string
		recipe_name = recipe_name.strip()

		#img_html = soup.find(id="main-photo").src.string
		img_html = soup.find(id="main-photo").img.get('src')
		if img_html == 'https://assets.cpcdn.com/assets/blank_logo_recipe_large.png?1b12035e517eeddc39631fb65f35fcad69e4b14bf9275634c54589903d659823':
			continue

		ingredient_name = ''
		ingredient_quantity = ''
		ingredients = soup.find(id = 'ingredients_list')
		for child in ingredients.children:
			if (str(child).find('ingredient_category') != -1):
				continue
			drop_html = re.compile(r'<[^>]+>',re.S)
			text = drop_html.sub('',str(child))
			text = text.strip()
			if text != '':
				#print (text.split())
				text = text.split()
				ingredient_name +=(text[0] + '\n')
				ingredient_quantity += (text[1] + '\n')

		drop_html = re.compile(r'<[^>]+>',re.S)
		date_reg_exp = re.compile('\d{4}[-/]\d{1,2}[-/]\d{1,2}')
		step_text = ''
		step_time = ''
		recipe_text = soup.find_all('dd')
		#print(recipe_text)
		for text_str in recipe_text:

			text = drop_html.sub('',str(text_str.find_all('p')))
			if (date_reg_exp.search(text) != None):
				#print('matched')
				continue

			text = text.replace("[", "")
			text = text.replace("]", "")
			if text != '':
				#print(text)
				text = text.strip()
				get_time = re.compile(r'\d{1,2}分')
				res = get_time.search(text)
				if res:
					step_time+=res.group()[:-1]+'\n'
				else:
					step_time+='0\n'
				step_text+=(text + '\n')

		cursor = db.cursor()
		try:
			sql = "insert into Recipe_recipe(name, image, material, amount, step, time) values('%s','%s','%s','%s','%s','%s')" % (recipe_name, img_html, ingredient_name, ingredient_quantity, step_text,step_time)
			cursor.execute(sql)

			db.commit()
			#print ('db')
		except:
			db.rollback()
			print ('db error')
	except Exception as e:
		print(e)
		error_number = error_number + 1
	else:
		download_number += 1
		if (download_number >= 100): break
cursor.close()


import requests
from bs4 import BeautifulSoup
import json

cats = [
    'Adult',
    'Arts',
    'Business',
    'Computers',
    'Games',
    'Health',
    'Home',
    'Kids_and_Teens',
    'News',
    'Recreation',
    'Reference',
    'Regional',
    'Science',
    'Shopping',
    'Society',
    'Sports',
]

pages = range(20)

data = []

for cat in cats:
    print('fetching cat: '+cat)
    for page in pages:
        print('page: #'+str(page))
        url = 'http://www.alexa.com/topsites/category;'+str(page)+'/Top/'+cat
        r = requests.get(url)
        html = r.text
        soup = BeautifulSoup(html)
        site_listing_elements = soup.find_all('li', class_='site-listing')
        for e in site_listing_elements:
            domain = e.a.text.lower()
            datum = [domain, cat]
            data.append(datum)

json_str = json.dumps(data)
f = open('alexa.json', 'w')
f.write(json_str)


"""
Implement the qmix algorithm with tensorflow, also thanks to the pymarl repo.

https://github.com/oxwhirl/pymarl  with pytorch implemented.

"""

import os
import sys
from types import SimpleNamespace as SN

import numpy as np

from absl import logging
# from attrdict import AttrDict
from xt.model.tf_compat import tf
from xt.algorithm import Algorithm
from xt.algorithm.algorithm import ZFILL_LENGTH
from xt.algorithm.qmix.episode_buffer_np import EpisodeBatchNP, ReplayBufferNP
from xt.algorithm.qmix.transforms import OneHotNp
from zeus.common.util.register import Registers

class DecayThenFlatSchedule(object):
    """Schedule from pymarl."""

    def __init__(self, start, finish, time_length, decay="exp"):

        self.start = start
        self.finish = finish
        self.time_length = time_length
        self.delta = (self.start - self.finish) / self.time_length
        self.decay = decay

        if self.decay in ["exp"]:
            self.exp_scaling = ((-1) * self.time_length / np.log(self.finish)
                                if self.finish > 0 else 1)

    def eval(self, t):
        """Schedule with eval times."""
        val = 0
        if self.decay in ["linear"]:
            val = max(self.finish, self.start - self.delta * t)
        elif self.decay in ["exp"]:
            val = min(self.start, max(self.finish, np.exp(-t / self.exp_scaling)))
        else:
            raise KeyError("invalid decay-{} configured".format(self.decay))
        return val

class EpsilonGreedyActionSelector(object):
    """Create epsilon greedy action selector from pymarl."""

    def __init__(self, args):
        self.args = args

        self.schedule = DecayThenFlatSchedule(
            args["epsilon_start"],
            args["epsilon_finish"],
            args["epsilon_anneal_time"],
            decay="linear",
        )
        self.epsilon = self.schedule.eval(0)

    def select_action(self, agent_inputs, avail_actions, t_env, test_mode=False):
        """Assume agent_inputs is a batch of Q-Values for each agent bav."""
        self.epsilon = self.schedule.eval(t_env)

        if test_mode:
            # Greedy action selection only
            self.epsilon = 0.0

        # mask actions that are excluded from selection
        # masked_q_values = agent_inputs.clone()
        # we make the `agent_inputs` value as a numpy !!!
        masked_q_values = agent_inputs.copy()
        # should never be selected!
        # masked_q_values[avail_actions == 0.0] = -float("inf")
        masked_q_values[avail_actions < 1e-6] = -float("inf")

        # random_numbers = th.rand_like(agent_inputs[:, :, 0])
        random_numbers = np.random.rand(*agent_inputs[:, :, 0].shape)
        # pick_random = (random_numbers < self.epsilon).long()
        pick_random = np.array(random_numbers < self.epsilon).astype(np.long)
        _batch_size, agent_num, action_dim = avail_actions.shape
        # avail_actions = avail_actions.reshape(-1, last_dim)
        avail_norm_p = np.array(
            avail_actions / np.expand_dims(avail_actions.sum(-1), -1)).astype(np.float)
        avail_norm_p = avail_norm_p.reshape(-1, action_dim)

        random_actions = np.array(
            [np.random.choice(action_dim, p=pi)
             for pi in avail_norm_p]).reshape((-1, agent_num))

        picked_actions = pick_random * random_actions + \
            (1 - pick_random) * masked_q_values.argmax(axis=2)
        # print(picked_actions, picked_actions.shape)
        # # [[5. 5.]] (1, 2)
        return picked_actions

@Registers.algorithm
class QMixAlg(Algorithm):
    """Implemente q-mix algorithm with tensorflow framework."""

    def __init__(
            self,
            model_info: dict,
            alg_config: dict,
            **kwargs,
    ):
        """
        Initialize.

        Consider the compatibility between trainer and explorer,
        QMix Algorithm could support the two scene
        """
        # avail_actions vary with env.map
        # alg_config = AttrDict(alg_config)
        logging.debug("get alg_config: {}".format(alg_config))

        env_info = alg_config["env_attr"]
        alg_config.update({
            "n_agents": env_info["n_agents"],
            "n_actions": env_info["n_actions"],
            "state_shape": env_info["state_shape"],
        })

        self.n_agents = alg_config["n_agents"]  # from env

        self.scheme = {
            "state": {
                "vshape": env_info["state_shape"]
            },
            "obs": {
                "vshape": env_info["obs_shape"],
                "group": "agents"
            },
            "actions": {
                "vshape": (1, ),
                "group": "agents",
                "dtype": np.int64
            },
            "avail_actions": {
                "vshape": (env_info["n_actions"], ),
                "group": "agents",
                "dtype": np.int32,
            },
            "reward": {
                "vshape": (1, )
            },
            "terminated": {
                "vshape": (1, ),
                "dtype": np.uint8
            },
            "actions_onehot": {
                "vshape": (env_info["n_actions"], ),
                "dtype": np.float32,
                "group": "agents",
            },
            # "filled": {"vshape": (1,), "dtype": tf.int64},
        }
        self.obs_shape = self._get_input_shape(alg_config, self.scheme)
        logging.debug("obs_shape: {}".format(self.obs_shape))

        logging.debug("update obs shape: {} --> {}".format(
            model_info["actor"]["model_config"]["obs_shape"], self.obs_shape))
        model_info["actor"]["model_config"]["obs_shape"] = self.obs_shape

        # NOTE: set graph scene, train as default
        model_info["actor"].update({"scene": kwargs.get("scene", "train")})

        super(QMixAlg, self).__init__(alg_name="QMixAlg",
                                      model_info=model_info["actor"],
                                      alg_config=alg_config)

        self.async_flag = False

        self.previous_state = None
        self.ph_hidden_states_in = None
        self.hidden_states_out = None

        self.params = None
        self.inputs = None
        self.out_actions = None

        self.avail_action_num = env_info["n_actions"]

        # use the episode limit as fix shape.
        self.fix_seq_length = env_info["episode_limit"]

        self.schedule = DecayThenFlatSchedule(
            alg_config["epsilon_start"],
            alg_config["epsilon_finish"],
            alg_config["epsilon_anneal_time"],
            decay="linear",
        )
        self.epsilon = self.schedule.eval(0)

        # select action
        self.selector = EpsilonGreedyActionSelector(alg_config)

        # # mix
        # self.state_dim = int(np.prod(alg_config.state_shape))
        # self.embed_dim = alg_config.mixing_embed_dim

        # update target with period, while update explorer each times.
        self.last_target_update_episode = -9999.0

        self.groups = {"agents": env_info["n_agents"]}
        preprocess_np = {"actions": ("actions_onehot",
                                     [OneHotNp(out_dim=alg_config["n_actions"])])}

        self.buffer = ReplayBufferNP(
            self.scheme,
            self.groups,
            alg_config["buffer_size"],
            env_info["episode_limit"] + 1,
            preprocess=preprocess_np,
        )
        self.train_batch = None
        self.train_times = 0

    @staticmethod
    def _get_input_shape(alg_config, scheme):
        """Assemble input shape with alg_config, vary with last_action/agent_id."""
        input_shape = scheme["obs"]["vshape"]
        if alg_config["obs_last_action"]:
            input_shape += scheme["actions_onehot"]["vshape"][0]
        if alg_config["obs_agent_id"]:
            input_shape += alg_config["n_agents"]
        return input_shape

    def reset_hidden_state(self):
        self.actor.reset_hidden_state()

    def build_inputs(self, batch, t):
        """
        Build inputs.

        # Assumes homogenous agents with flat observations.
        # Other MACs might want to e.g. delegate building inputs to each agent
        1. inference stage, use batch = 1,
        2. train stage, use batch = episode.limit

        Also, use numpy for combine the inputs data
        """
        _batch_size = batch.batch_size
        inputs = list()
        inputs.append(batch["obs"][:, t])  # b1av

        if self.alg_config["obs_last_action"]:
            if t == 0:
                inputs.append(np.zeros_like(batch["actions_onehot"][:, t]))
            else:
                inputs.append(batch["actions_onehot"][:, t - 1])

        if self.alg_config["obs_agent_id"]:
            _ag_id = np.expand_dims(np.eye(self.n_agents), axis=0)  # add axis 0
            inputs.append(np.tile(_ag_id, (_batch_size, 1, 1)))  # broadcast_to

        inputs = np.expand_dims(np.concatenate(inputs, axis=-1), axis=1)

        return inputs

    def predict_with_selector(self, ep_batch, t_ep, t_env, test_mode):
        avail_actions = ep_batch["avail_actions"][:, t_ep]
        agent_inputs = self.build_inputs(ep_batch, t_ep)

        out_val = self.actor.infer_actions(agent_inputs)

        select_actions = self.selector.select_action(
            out_val, avail_actions, t_env, test_mode=test_mode)
        # print("out_val: {}, select action: {}, avail_actions, {}, t_env:{}".format(
        #     out_val, select_actions, avail_actions, t_env))
        return select_actions

    def save(self, model_path, model_index):
        """Save qmix explore agent weight with saver."""
        model_name = os.path.join(
            model_path, "actor{}".format(str(model_index).zfill(ZFILL_LENGTH)))
        self.actor.save_explore_agent_weights(model_name)
        return [model_name]

    def restore(self, model_name=None, model_weights=None):
        """
        Restore the model with the priority: model_weight > model_name.
        owing to actor.set_weights would be faster than load model from disk.

        if user used multi model in one algorithm,
        they need overwrite this function.
        And, Caller make the name/weights valid.
        """
        if model_weights is not None:
            self.actor.set_weights(model_weights)
        else:
            logging.debug("{} try load model: {}".format(self.alg_name, model_name))
            self.actor.restore_explorer_variable(model_name)

    @staticmethod
    def _new_data_sn():
        new_data = SN()
        new_data.transition_data = {}
        new_data.episode_data = {}
        return new_data

    def prepare_data(self, train_data, **kwargs):
        """Insert trajectory into buffer, and sample batch if meet required."""
        new_data = self._new_data_sn()
        # received train data batch size ==1, seq_length==limit episode
        for k, val in train_data.items():
            new_data.transition_data[k] = val

        deliver_batch = EpisodeBatchNP(
            self.scheme, self.groups, 1, self.fix_seq_length + 1, data=new_data)
        self.buffer.insert_episode_batch(deliver_batch)

        if self.buffer.can_sample(self.alg_config["batch_size"]):
            self.train_batch = self.buffer.sample(self.alg_config["batch_size"])
        else:
            self.train_batch = None

    def train(self, **kwargs):
        """Train with buffer sampled."""
        # , batch: EpisodeBatchNP, t_env: int, episode_num: int
        if not self.train_batch:
            return np.nan

        episode_num = kwargs.get("episode_num")
        if not episode_num:
            raise KeyError("need episode num to update target network")

        batch = self.train_batch

        # Truncate batch to only filled timesteps
        max_ep_t = batch.max_t_filled()
        logging.debug("episode sample with max_ep_t: {}".format(max_ep_t))
        # batch = batch[:, :max_ep_t]

        # Get the relevant quantities
        rewards = batch["reward"][:, :-1]
        actions = batch["actions"][:, :-1]
        terminated = batch["terminated"][:, :-1].astype(np.float32)
        mask = batch["filled"][:, :-1].astype(np.float32)
        mask[:, 1:] = mask[:, 1:] * (1 - terminated[:, :-1])
        avail_actions = batch["avail_actions"]

        # [bs, seq_len, n_agents, obs_size] [32, 1, 2, 26] --> [32, 301, 2, 26]
        _inputs = [self.build_inputs(batch, t) for t in range(batch.max_seq_length)]

        batch_trajectories = np.concatenate(_inputs, axis=1)

        logging.debug("batch_trajectories.shape: {}".format(batch_trajectories.shape))
        logging.debug("rewards.shape: {}".format(rewards.shape))
        logging.debug("actions.shape: {}".format(actions.shape))
        logging.debug("terminated.shape: {}".format(terminated.shape))
        logging.debug("mask.shape: {}".format(mask.shape))
        logging.debug("avail_actions.shape: {}".format(avail_actions.shape))
        logging.debug("batch.max_seq_length: {}".format(batch.max_seq_length))
        logging.debug("batch.batch_size: {}".format(batch.batch_size))

        # to get action --> [32, 300, 2, 7]
        # [32*301*2, 26] --> [32*301*2, 7] --> [32, 301, 2, 7] --> [32, 300, 2, 7]
        # batch4train = batch_trajectories.reshape([-1, batch_trajectories.shape[-1]])

        self.train_times += 1
        loss_val = self.actor.train(
            batch_trajectories,
            list([max_ep_t for _ in range(batch.batch_size * self.n_agents)]),
            avail_actions,
            actions,
            batch["state"][:, :-1],
            batch["state"][:, 1:],
            rewards,
            terminated,
            mask,
        )

        # update explore agent after each train processing.
        self.actor.assign_explore_agent()

        # update target network as required.
        if (episode_num - self.last_target_update_episode) / self.alg_config["target_update_interval"] >= 1.0:
            self.actor.assign_targets()
            logging.debug("episode-{}, target Q network params replaced!".format(episode_num))
            print("episode-{}, target Q network params replaced!".format(episode_num))
            print(">>> train-{} use seq-len-{}".format(self.train_times, max_ep_t))
            sys.stdout.flush()
            self.last_target_update_episode = episode_num

        return loss_val

    def train_ready(self, elapsed_episode, **kwargs):
        """
        Support custom train logic.

        :return: train ready flag
        """
        # we set train ready as default
        # if elapsed_episode < self.alg_config["batch_size"]:
        if not self.buffer.can_sample(self.alg_config["batch_size"]):
            self._train_ready = False
            if not kwargs.get("dist_dummy_model"):
                raise KeyError("qmix need to dist dummy model.")
            # dist dummy model
            kwargs["dist_dummy_model"]()
        else:
            self._train_ready = True

        return self._train_ready


"""Train patient's appointment no show model."""
import multiprocessing
from datetime import datetime

import numpy as np
from pandera.decorators import check_types
from pandera.typing import DataFrame
from xgboost import XGBClassifier

from patient_no_show.schema import (
    PatientsAppointmentFeaturesLabel,
    PatientsAppointmentFeaturesLabelPrediction,
)

@check_types
def train_no_show_model(
    *,
    patient_appointments: DataFrame[PatientsAppointmentFeaturesLabel],
    validation_start: datetime,
) -> tuple[XGBClassifier, DataFrame[PatientsAppointmentFeaturesLabelPrediction]]:
    """Train no_show model on provided data and time slice.

    Model is trained in a two step approach:
    1. Train early stopping on data before validation_start. Predict on validation data and return.
    2. Retrain with early stopping results (n_estimators) and return model.

    Args:
        patient_appointments (DataFrame[PatientsAppointmentFeaturesLabel]): features with label
        validation_start (datetime): timestamp for splitting into train and validation data

    Returns:
        tuple[XGBClassifier, DataFrame[PatientsAppointmentFeaturesLabelPrediction]]: model and
            artificial model prediction with early stopping model on all data.
    """
    # train/validation split
    train_data = patient_appointments.query(f"appointment_day <= '{validation_start}'").reset_index(
        drop=True
    )
    validation_data = patient_appointments.query(
        f"'{validation_start}' < appointment_day"
    ).reset_index(drop=True)

    # feature definition
    PATIENT_NO_SHOW_MODEL_FEATURES = [
        "gender",
        "age",
        "neighborhood",
        "scholarship",
        "hipertension",
        "diabetes",
        "alcoholism",
        "handcap",
        "sms_received",
        "days_between_scheduling_and_appointment",
        "appointment_scheduling_dayofweek",
        "appointment_scheduling_hour",
    ]

    # first model training with early stopping on training set only
    model = XGBClassifier(
        **{
            "booster": "gbtree",
            "objective": "binary:logistic",
            "tree_method": "approx",
            "enable_categorical": True,
            "n_estimators": 5000,
            "colsample_bytree": 0.9,
            "learning_rate": 0.2,
            "max_depth": 6,
            "min_child_weight": 5,
            "eval_metric": "logloss",
            "early_stopping_rounds": 100,
            "n_jobs": int(multiprocessing.cpu_count() / 2),
        }
    )

    eval_set = [
        (
            train_data.filter(items=PATIENT_NO_SHOW_MODEL_FEATURES),
            train_data.no_show,
        ),
        (
            validation_data.filter(items=PATIENT_NO_SHOW_MODEL_FEATURES),
            validation_data.no_show,
        ),
    ]

    model.fit(
        X=train_data.filter(items=PATIENT_NO_SHOW_MODEL_FEATURES),
        y=train_data.no_show,
        eval_set=eval_set,
        verbose=10,
    )
    # true index column index from trained model
    # https://xgboost.readthedocs.io/en/latest/python/python_api.html#xgboost.XGBClassifier.predict_proba
    true_predict_proba_index = np.nonzero(model.classes_)[0].item()
    # artificial model prediction on validation data
    validation_data_prediction = validation_data.assign(
        no_show_probability_prediction=lambda df: model.predict_proba(
            X=df.filter(items=PATIENT_NO_SHOW_MODEL_FEATURES)
        )[:, true_predict_proba_index],
        no_show_class_prediction=lambda df: df.no_show_probability_prediction > 0.5,
    )

    # retrain model on all data
    # OPTIONAL: cleaner distinction between phase 1 and phase 2 models
    model.n_estimators = round(model.best_ntree_limit * 1.2)  # train 20% longer, due to more data
    model.early_stopping_rounds = None  # remove early stopping
    model.fit(
        X=patient_appointments.filter(items=PATIENT_NO_SHOW_MODEL_FEATURES),
        y=patient_appointments.no_show,
    )
    return model, validation_data_prediction


from pycaret.regression import *
from rdkit import Chem
from rdkit.Chem import Descriptors
from rdkit.Chem import Lipinski
from rdkit.Chem import Crippen
import numpy as np
import streamlit as st
import PIL.Image

def getAromaticProportion(m):
    aromatic_list = [m.GetAtomWithIdx(i).GetIsAromatic() for i in range(m.GetNumAtoms())]
    aromatic = 0
    for i in aromatic_list:
        if i:
            aromatic += 1
    heavy_atom = Lipinski.HeavyAtomCount(m)
    return aromatic / heavy_atom

def generate(smiles):
    moldata = []
    for elem in smiles:
        mol = Chem.MolFromSmiles(elem)
        moldata.append(mol)

    baseData = np.arange(1, 1)
    i = 0
    for mol in moldata:

        desc_MolLogP = Crippen.MolLogP(mol)
        desc_MolWt = Descriptors.MolWt(mol)
        desc_NumRotatableBonds = Lipinski.NumRotatableBonds(mol)
        desc_AromaticProportion = getAromaticProportion(mol)

        row = np.array([desc_MolLogP,
                        desc_MolWt,
                        desc_NumRotatableBonds,
                        desc_AromaticProportion])

        if i == 0:
            baseData = row
        else:
            baseData = np.vstack([baseData, row])
        i = i + 1

    columnNames = ["MolLogP", "MolWt", "NumRotatableBonds", "AromaticProportion"]
    descriptors = pd.DataFrame(data=baseData, columns=columnNames)

    return descriptors

model = load_model("WaterSolubility 19-JAN-21")

########################################################################################################################
#Creating Graphical User Interface
########################################################################################################################

st.write("""
    # Molecular Solubility Prediction Web App
    This WebApp predicts the **Solubility (LogS)** values of molecules!""")
st.image(PIL.Image.open("banner-project.jpg"), width=600)

st.header("Input Smile Format of Molecule\n Multiple Entries are followed by new line")
SMILES_input = "NCCC\nCN"
SMILES = st.text_area("SMILES input", SMILES_input)
SMILES = "C\n" + SMILES  # Adds C as a dummy, first item
SMILES = SMILES.split('\n')

st.header('Input SMILES')
st.dataframe(SMILES[1:])

st.header('Computed molecular descriptors')
X = generate(SMILES)
st.dataframe(X[1:])

st.header("Computed Molecular Solubility(LogS)")
prediction = model.predict(X[1:])
st.dataframe(prediction)


import concurrent.futures

from utils.fileHandler import isFileAvailable
from utils.fileHandler import writeJSONToFile
from utils.fileHandler import readJSONFromFile

CONNECTIONS = 10
TIMEOUT = 5

def execFctInParallel(arr, fct):
  ## Init
  i = 0
  total = len(arr)

  ## Exec concurrently
  with concurrent.futures.ThreadPoolExecutor(max_workers=CONNECTIONS) as executor:
    future_to_url = (executor.submit(fct, el, TIMEOUT) for el in arr)

    ## Get results
    output = []
    for future in concurrent.futures.as_completed(future_to_url):
      try:
        data = future.result()
        output.append(data)
        # print("load completed for {}".format(data.get('id')))
        i = i + 1
        print("progress {}/{}".format(i, total))
      except Exception as exc:
        print("ERROR")
        data = str(type(exc))
      finally:
        # output.append(data)
        print(str(len(output)),end="\r")
    return output

def storeParallelOuputToFile(output, filename):
    store = {}
    if(isFileAvailable(filename)):
        store = readJSONFromFile(filename).get("data")
    for o in output:
        oId = o.get('id')
        store[oId] = o.get('output')
    writeJSONToFile(filename, store)
    return store

def massObjectUpdate(arr, filename, fct, update = False):
  ## Initialize
  store = {}
  if(isFileAvailable(filename)):
    store = readJSONFromFile(filename).get("data")

  ## Identify ids that have to be updated
  idsToUpdate = []
  if(update):
    for o in arr:
      oId = str(o.getId())
      idsToUpdate.append(oId)
  else:
    keysInFile = []
    for key, val in store.items():
      keysInFile.append(key)
    # print("Keys: {}".format(keysInFile))
    for o in arr:
      oId = str(o.getId())
      if(not (oId in keysInFile)):
        idsToUpdate.append(oId)
  # print("IDs to update: {}".format(idsToUpdate))

  ## Fetch additional information and store them in file
  output = execFctInParallel(idsToUpdate, fct)
  store = storeParallelOuputToFile(output, filename)

  ## Enrich object with information just fetched
  for o in arr:
    oId = str(o.getId())
    o.setDetailsJSON(store[oId])

  return arr


from xml.etree.ElementTree import Element, ElementTree, SubElement, dump

def indent(elem, level=0): #자료 출처 https://goo.gl/J8VoDK
    i = "\n" + level*"  "
    if len(elem):
        if not elem.text or not elem.text.strip():
            elem.text = i + "  "
        if not elem.tail or not elem.tail.strip():
            elem.tail = i
        for elem in elem:
            indent(elem, level+1)
        if not elem.tail or not elem.tail.strip():
            elem.tail = i
    else:
        if level and (not elem.tail or not elem.tail.strip()):
            elem.tail = i

def makeADSBXmlFile():
    root = makeUAM()

    ElementTree(root).write("ADSTestData/ADSTestData.xml")

def makeUAM():
    root = Element("UAM")

    makeFlightIdentifier(root)
    makeCurrentTime(root)
    makeCurrentPosition(root)
    indent(root)
    dump(root)

    return root

def makeFlightIdentifier(root):
    node = SubElement(root, "flightIdentifier")

    makeUAMIdentifier(node)

def makeUAMIdentifier(node):
    node2 = Element("uamIdentification")

    node2.text = "UAL123"

    node.append(node2)

def makeCurrentTime(root):
    node = SubElement(root, "currentTime")

    SubElement(node, "Date").text = "111"
    SubElement(node, "Time").text = "222"
    SubElement(node, "TimeReference").text = "333"

def makeCurrentPosition(root):
    node = SubElement(root, "currentPosition")

    makeLocation(node)

def makeLocation(node):
    location_node = Element("location")

    SubElement(location_node, "longitude").text = "126.9525465"
    SubElement(location_node, "latitude").text = "37.5453577"

    node.append(location_node)

if __name__ == "__main__":
    makeADSBXmlFile()


from django.http import HttpResponse
import datetime
from django.template import Template, Context, loader
from django.template.loader import get_template
from django.shortcuts import render

class Persona(object):
  def __init__(self, nombre, apellido):
    self.nombre=nombre
    self.apellido=apellido

def saludo(request):
  p1=Persona("Mario", "Ramirez")
  temas=["item1", "item2", "item3", "item4", "item5", "item6"]
  fecha_actual=datetime.datetime.now()

  #doc_externo = get_template("plantillauno.html")
  #documento=doc_externo.render({"name": p1.nombre, "apellido": p1.apellido, "fecha_actual": fecha_actual, "temas": temas})
  return render(request, "plantillauno.html", {"name": p1.nombre, "apellido": p1.apellido, "fecha_actual": fecha_actual, "temas": temas})

def antiguedades(request):
  return render(request, "Antiguedades.html", {"name_person": "pedrito"})

def saludar(request):
  p1=Persona("Juan", "Perez")
  temas=["item1", "item2", "item3", "item4", "item5", "item6"]
  fecha_actual=datetime.datetime.now()
  doc_externo=open("C:/Users/natik/Dropbox/Fullstack/projects/django/categories_django/categories_django/templates/plantillauno.html")
  plt=Template(doc_externo.read())
  doc_externo.close()
  ctx=Context({"name": p1.nombre, "apellido": p1.apellido, "fecha_actual": fecha_actual, "temas": temas})

  documento=plt.render(ctx)

  return HttpResponse(documento)

def despedida(request):
  return HttpResponse("Adios Amig@")

def getDate(request):
  fecha_actual=datetime.datetime.now()
  documento="""

Fecha y hora actuales %s

    """ % fecha_actual
  return HttpResponse(documento)

def getAge(request, edad, year):
  periodo = year - 2020
  edadFutura = edad + periodo
  documento = """

En el año %s tendrás %s años

    """ %(year, edadFutura)
  return HttpResponse(documento)